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Abstract. 



In this paper, we investigate a unique method of inventing linear edge enhancement 
operators using evolution and reconfigurable hardware. We show that the technique is 
motivated by the desire for a totally automated object recognition system. We show that 
an important step in automating object recognition is to provide flexible means to smooth 
images, making features more obvious and reducing interference. Next we demonstrate 
a technique for building an edge enhancement operator using evolutionary methods, 
implementing and testing each generation using the Xilinx 6200 family FPGA. Finally, 
we present the results and conclude by mentioning some areas of further investigation. 



Introduction 

Image edge enhancement is an important part of modern computerized object recognition 
methods. Edge enhancement typically requires convolution operators that produce weighted 
average transformations on individual pixels of some source image. Edge enhancement 
operators tend to be developed for particular types of problems and larger operators tend to be 
for very specific purposes. Large irregular pixel patterns require rather large and irregular 
edge enhancement operators. 

The smoothing process assists an edge detection algorithm to distinguish between 
background noise and actual objects in an image. 

Unfortunately, software implementations of large convolutions are extremely slow. 
Consequently, convolutions of any respectable size are typically implemented in specialized 
image processing hardware. Another problem with large convolutions is that it is often very 
difficult to "discover" good ones. The process of creating these convolution operators requires 
trial and error, and experience. 

Recently, at New Light Industries, we needed special purpose convolution operators for use 
in low-cost, commercial off-the-shelf hardware devices. These convolutions needed to be very 
efficient and implementable in reconfigurable hardware. Using genetic algorithms and Xilinx 
XC6000 technology, we implemented an evolutionary system that designed convolution 
operators for our application. The genetic algorithm evaluated operators by comparing the 
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convolution of one original image with several prepared images that the convolution should be 
able to produce. 

These experiments demonstrated both the viability, and some limitations, of creating edge 
enhancing convolution kernels using evolutionary techniques and prepared images. This paper 
describes our results and shows an evolved convolution that was used in an edge detecting 
algorithm. Finally, this paper presents some potential future research directions and 
applications. 



Motivation 

Computerized image processing provides many opportunities for implementing 
evolutionary problem-solving techniques. Our problem is to identify important features in a 
diffraction pattern that changes over time. This is analogous to identifying and tracking 
multiple objects moving through a series of frames. Consider a camera aimed at a moving 
object or an object illuminated by a moving light source. The problem is to identify the object 
as it moves through a series of still frames. Since we needed compact, inexpensive, efficient 
hardware to perform the identification, we chose to implement our solution in reconfigurable 
systems using XC6000 FPGAs from Xilinx. 

Our target images were a series of diffraction patterns recorded over time with a moving 
light source. The diffraction patterns are optically derived Fourier Transforms of features on 
the surface of an object. The frames are captured in 8-bit in gray-scale and the objects to be 
recognized are geometric "blobs" that both move and change shape from frame to frame. We 
must identify each object as it moves and changes shape over a series of frames. We chose a 
simple edge-detection algorithm [8] as the method for extracting the border. 

A key step in identifying object borders in a particular frame is applying a large 
convolution operator to the image. We used genetic methods to develop appropriate 
convolution operators for our training set. The following sections explain the approach and 
the results of our initial experiments. 



Background 



Image Processing Considerations 

The literature on image processing contains many examples and techniques for performing 
image enhancement and image segmentation. Generally, as one might expect with any 
computational task, there are tradeoffs between speed and accuracy in the segmentation 
process. We used optically generated Fourier Transforms of surface features for this 
experiment. Identification of features in the frequency space of these images amounted to 
identifying the object being scanned. We needed to identify not just the features on many 
images, but the path of each feature through multiple consecutive frames. We investigated 
some well-known methods for image detection that we could use to facilitate segmentation 
and object tracking. 

For the edge detection method, we looked at three algorithms: Marr-Flildreth [8], the 
Infinite Symmetric Exponential Filter (ISEF) [9], and morphological boundary extraction [10]. 

Marr-Flildreth edge detectors use a smoothing algorithm, usually a convolution with a 
Gaussian function, followed by a zero-crossing binarization of the image. We chose this 
method because it is simple, fast, and easy to implement in varied hardware and software. 

The ISEE is a very high quality edge detector on images with high frequency uniform 
noise. The ISEE also computes zero crossings to find edges, but it performs a much more 
complex smoothing using a band-limited Laplacian operator. The ISEF algorithm we 
implemented [11] used an edge following technique that consumed memory and probably 
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could have been time-optimized as well. We found this algorithm to be very effective, but 
much slower than Marr-Hildreth. Since the latter was adequate for our task, we did not use 
the ISEF. 

Morphological Boundary Extraction uses some simple set operations to outline edges on 
the boundaries of objects in an image. In the simplest case, the edge enhancement results 
from eroding a copy of the image using a simple kernel and then subtracting the resulting 
image from the original image. An image containing high frequency noise requires an 
additional opening step to clean the frame before the boundary extraction will work correctly. 
This algorithm was not as effective for extracting images from our images. The boundaries 
of different features in the image tend to bleed together if the erosion kernels are not of the 
proper size for the boundary. If the boundary is different in different parts of the image it may 
be necessary to erode different parts of the image with different kernels to get an accurate 
boundary. 

We also investigated some thresholding methods, but found them to be very limited. One 
method we did not explore which might prove effective is Edge Level Thresholding, though 
this method proves to be slow relative to the methods we chose to investigate. 

We decided to use the Marr-Hildreth method initially, because it was by far the fastest 
method given our environment. The ISEF technique proved more accurate in object 
segmentation, but the improvement was not warranted given that the process was significantly 
slower than Marr-Hilreth. The Morphological technique we investigated worked adequately 
only with a great deal of “hand-tweaking” the parameters. This actually made the 
morphological boundary extraction method a candidate for later experiments with genetic 
methods. For the current experiment, though, we found it difficult to control the results when 
we tried to automate the algorithm. 



Convolution 



Convolution operators are commonly used in edge enhancement techniques to increase 
contrast or emphasize features with particular shapes on digitized images. "Edges" are usually 
rapid changes in pixel values. One way to think about a convolution is as a differential 
operator that measures the rate of change in some direction along the image. Convolution 
operators for image processing are usually expressed as odd-valued square matrices. The 
operation is applied across the pixel field by multiplying individual pixel values by the matrix 
elements and then adding the results to get a new pixel value. The new value replaces the 
value for the center pixel in the field. The operation is applied repeatedly across the picture 
until the whole pixel field has been transformed. 

Simple convolutions are often 3x3 or 5x5 matrices. One simple and straightforward 
convolution is the "directionless" or Laplacian operator. The Lapacian operator is directly 

v72 

analogous to the Laplacian of differential analysis: V = — — H — 

dx dy 

Using the definition of a difference, f'~{f{x) — f{x — h))lh, and letting h=l, the 



Laplacian can be expressed discretely as a 3x3 matrix as follows: 

To -1 0“ 

-1 4 -1 

0-10 



This is only one common example of a convolution matrix. We will show examples created 
by genetic methods later in this paper. 

When the convolution is applied to a 3x3 area on a pixel field, the operator gives a new 
value for the center pixel in a transformed image. The convolution is then moved across and 
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down the pixel field until all the pixels have been converted. At the edges, there are a number 
of different techniques for operating on pixels that cannot be mapped to the center of a 3x3 
matrix. In our experiments, we ignored the values at the borders of the pixel fields. 

The convolution operation, though quite simple, can produce complex and often inscrutable 
results. We present a little example of convolution applied to a small array below. Suppose 
one has a section of a gray scale image such as that in Figure 1. This corresponds to some 
portion of an image. 
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Figure la. A section of a gray scale image. The convolution operates on incremental 
sections of the image, beginning with the highlighted section. Figure lb. The result of 
the convolution with Laplacian of image la. Note that the edges of the image are 0 
padded. Also any convolution that sums to a negative number is set to zero. 



Convolution is defined as the sum of the element-by-element products of two matrices. So 
in the case of the Laplacian operator, and an NxN matrix, the convolution describes a new 

X — ' 1 X — ' 1 

matrix whose elements are defined as: A* A, , = 



I' s: 



'.7 

After moving this convolution operator across the shaded area of the image above, one 
continues with the next 3x3 section of the matrix. Note how the convolution "enhances" one 
section of the matrix associated with local maxima that are not on edges. 

The Laplacian is only one of many widely used convolution operators (or kernels). Usually, 
given a particular shape to recognize and a particular scale, there is a particular class of 
convolution kernels that will be most effective in enhancing the image to recognize features of 
the desired shape or scale. When shapes are large or irregular, it can be difficult to develop an 
effective (or efficient) convolution kernel to recognize the shape. 



Evolutionary Computing with XC6000 FPGAs 

We performed the image extraction and original image processing testing using a Matrox 
Genesis image grabber, which includes a TMS320C80 with parallel fixed-point MACs and a 
32-bit data bus, allowing us to very handily evaluate the different edge detection methods 
discussed above. At the time of its purchase, the device approached $8,000 US, far from the 
COTS criteria that we were looking for. For this reason, and to facilitate the evolutionary 
aspect of our project, we chose the VCC Fat HOT PCI card for our development system. It 
was less than half the cost while providing the necessary computing power. 

Developing programs for FPGAs using evolutionary methods requires some attention to the 
special constraints of the reconfigurable hardware. FPGA-based designs, evolutionary or not, 
are physically constrained by geometry and function unit capabilities. Function units within 
any FPGA provide basic binary logical operations (e.g., OR, AND, XOR), but beyond this, 
there is a wide variation in the particular functions that can be implemented within a single 
function unit on the FPGA. 

Similarly, the geometry of the FPGA as a whole has an effect on design. The way in which 
individual function units are connected to other function units and to the edges of the device 
also constrains the routing of circuitry in the FPGA. We have chosen the Xilinx XC6000 
series of FPGAs for our research for the following reasons: 

• There is a significant body of research on evolutionary hardware design building around 

this class of FPGA devices. 
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• The chip design is open and configuration formats are accessible at the hardware level. 

• Function unit routing is extremely flexible. 

• The device has a well-defined PCI bus interface standard from the manufacturer; and this 
makes the device useful for experimenting with desktop computer interfaces in a consistent 
manner with other researchers and manufacturers. 

• The manufacturer has developed API sets that make device programming easily accessible 
to the software developer using a desktop PC. Other API sets are available for different 
development environments. 

The XC6000 gives us the flexibility to experiment with different evolutionary methods and 
to rapidly build and utilize new tools for our research. Many researchers describe the XC6200 
architecture as it applies to our experiment. In particular, see [3], [4] and [5]. Many possible 
approaches exist for evolutionary design using FPGAs. We will discuss relevant approaches to 
one feature recognition subsystem that have emerged from our present research. What 
distinguishes different evolutionary design approaches is the representation and evaluation of 
trial solutions. 



Evolutionary Hardware Design 



Representation 

The key to problem solving with genetic algorithms is to find a suitable representation of 
the problem. A good representation must encompass the full range of possible solutions, and 
must enable the genetic algorithm to generate new trial solutions with a relatively high 
probability of retaining beneficial features of old trial solutions. Mutation operators should 
change trial solutions in ways that lead to relatively small fitness changes, and recombination 
operators should tend to accumulate useful features of two or more trial solutions into a new 
trial solution. We investigated two different representations of FPGA configurations. One 
method uses a coded representation of possible functions and a coded representation of 
possible routings. 
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Figure 2: Coded Representation Implemented as a Look-Up Table. 
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Coded representations are common (see [4], [6]) in evolutionary programming, partly 
because they are relatively easy to describe. In our representation, our chromosome is a 2D 
array of integers that defined a convolution kernel numerically. The fitness evaluation step 
converts the kernel into a set of bit values and then converts the bit values into a configuration. 
The configuration performs the convolution but allows us to hard-code (or, more accurately, 
hard-wire) the convolution kernel into the circuit description. This reduces the circuit size and 
speeds up the evaluation of the convolution. 

The primary advantage of this representation is that it is abstracted from the particular 
FPGA architecture, so that it can be rewritten to configure different devices (as long as only 
common function unit/routing configurations are allowed). The primary disadvantages of this 
representation are: 

• Only common FPGA function configurations are allowed, so special features of certain 
FPGAs (i.e., separable clocking, register protection in the XC6000) will not be available to 
the evolutionary process. Note that this can be an advantage in some circumstances, such as 
when it is desirable to constrain the FPGA to clocked circuits. 

• The added level of abstraction (beyond a bit-level representation) adds a level of 
complexity to the testing process. 

We also evaluated intrinsic evolution, in which we directly use the underlying stmcture of 
the XC6000 in the function unit encoding. (For examples of intrinsic evolution, see [3].) In 
this representation we pay particular attention to the address/data structure of the FPGA 
configuration. Programming the FPGA at this level is analogous to programming a standard 
CPU using native machine code. The complexity of creating working programs at this level is 
avoided in the evolutionary process. FPGA configurations evolve to approach the problem 
represented in the fitness stage. 



Byte 0 


Output Routing 


Bytel 


Function Config/ Input 


Byte 2 


Function Config/ Input 



Figure 3: Function Unit Configuration Bytes 

A circuit configuration is represented as a string of bytes that together define the 
configuration of a function unit on the XC6000. The configuration of a single function unit 
comprises three bytes that define the local routing and the logic implemented in the function 
unit. 

Our chromosome is a two-dimensional array of these three-byte units. Each element of the 
array represents a function unit. The index of each element represents the relative position of 
the function unit within the FPGA, thereby allowing us to construct an address for each 
element of the configuration. For details about what particular values indicate, see [1]. The 
primary advantages of this representation are: 

• The representation may be directly interpreted as a "program" of the FPGA. Very little 
preliminary processing is needed to create a loadable configuration. 

• All possible functions are allowed in principle, but they can be restricted at will by masking 
the appropriate bits of each function unit configuration. 

The primary disadvantages of an intrinsic representation are: 

• The resulting evolved configuration can be very difficult for a human FPGA programmer to 
understand. 

• The configuration is always specific to a particular FPGA. A program generated via this 
method for one manufacturer’s FPGA cannot be expected to run on a different 
manufacturer’s FPGA. 
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The first representation discussed above is superior to the second for a number of practical 
reasons. Using the first method, we will not be tied to any particular FPGA for the purposes of 
development. It is, however, much more time-consuming to implement the first, so our 
experiments to date have concentrated on the second representation. There are, of course, 
other methods for creating FPGA configurations (See [6] in particular), but our methods are 
simple and adequate for the task. 



Fitness Evaluation and Crossover 

The evaluation of each solution created by the genetic algorithm is another key aspect of 
evolutionary design of programs for our experiment. We test each trial solution created by our 
genetic algorithm, and use the test results to assign a fitness value to the trial solution. 
Generally, the objective of the design process is to develop a design that meets certain 
performance criteria. In evolutionary design, however, it is important to use fitness values to 
guide evolution efficiently towards the final objective. 

The basic flow of our FPGA evolution algorithm is: 

1. Load a file that represents all the static configuration portions of the program (e.g., I/O 
Registers, Counters, and other control sections) 

2. Create a random population of trial solutions. 

3. For each trial solution, evaluate as follows: 

a. Convert the trial solution to a loadable configuration. 

b. Load the configuration. 

c. Load the input registers. 

d. Wait for some fixed amount of time. 

e. Read the output register. 

f. Compare the register with a calculated result based on the input to get the error. 

g. Assign the fitness of the trial solution based on step f. 

4. Select trial solutions according to fitness to generate a new population. 

5. Generate new trial solutions by applying genetic operators (mutation and recombination) to 
the selected trial solutions. 

6. Apply steps 3 through 6 repeatedly until a satisfactory solution emerges. 

In step g above, changes can be made to emphasize or de-emphasize certain aspects of the 
calculated error at different stages in the evolution so that, for example, basic functions are 
evolved and optimized before they are combined into more complex structures. Fitness 
calculation can include weighting the error to give more influence to solutions that meet 
certain constraints or sub-criteria. This is extremely important in the case of our application 
since we will be comparing vector values representing patterns of features identified by the 
FPGA to reference vectors calculated by other means. 

In order to maintain the integrity of working portions of each genome, we developed a 
crossover operator that preserves rectangular portions of the genome during crossover. By 
default, the 2D integer (3D Binary) genome in Gail will perform byte-level crossover. This 
has the effect of destroying local optimal areas in the genome from generation to generation. 
We tried to preserve local optimal behavior in the genome by treating each genome as a 2D 
array and each crossover component as a 2D sub-array of the genome. 

For example, given two 9x9 genomes to cross over, the cross-over algorithm works as 
follows: 

1. Randomly select a starting row and starting column. This will be the upper left corner of 
the crossover array. 

2. Randomly select the number of rows and columns to cross over. This defines the size of the 
crossover array. 
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3. Select two genomes to cross over and remove a sub-array from the appropriate position and 
of the calculated size, from each of the genomes. 

4. Exchange these sub-arrays between the genomes. 

5. The evolved convolutions were created by genetically developing a convolution to attempt 
to "reproduce" an image that was enhanced with a general-purpose convolution (a pseudo- 
Laplacian) in special purpose image processing hardware. 

We evaluated fitness as follows: 

1. Get a genome. 

2. Convolve the original image with the genome. 

3. Compare the resulting image with the image convolved with the 9x9 pseudo-Laplacian 
(See Figure 4 below) to get the fitness value. The evaluation compares each pixel in the 
resulting image with each pixel in the original image. 

4. Compare fitnesses preserving the lowest value (highest fitness) individuals. 



Results 

The experiments we performed were carried out using a genetic algorithm defined using 
GALIB 2.4.2 and the XILINX API library for the 6200 parts. The GA was set up to evolve a 
9x9 2-dimensional integer genome. We ran the GA with population size of 50, crossover 
probability of 20%, and mutation probability of 5%. We noticed that, under these conditions, 
the GA produced very stable results after approximately 100 generations. 

As we see in the pictures below, low fitness members of the population actually produced 
"terrible" resulting images. We determined that this was due to numeric overflow during 
fitness evaluation. Because of this the fitness evaluation was re-examined and we decided to 
reward fitness values close to half the maximum value of a long integer. One "good" result is 
shown in Figure 6. 

We present some examples of evolved convolutions and the enhanced images they 
produced below. The GA created and evaluated these images using the training set image in 
Figure 4. 
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0 000 0 000 0 

0 000 0 000 0 

0 000 0 000 0 

-1 000 8 000 -1 

0 000 0 000 0 
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Figure 4: An original diffraction pattern from the training set (left), a pseudo 
Laplacian operator(right) and the convolved image used for comparison (center). 

Evolved convolution operators produce some widely varying results. Below are a few 
examples of evolved operators and the images they produced from the original image in 
Eigure 4. 
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Evolved convolution and enhanced image Evolved convolution and enhanced image 
with fitness = 12644. with fitness = 252040. 



Figure 5: Two High Fitness Results 

In the "high-fitness" convolution operators, we can see what appears to be a random 
distribution of pixel values through the dark areas of the original frame. These operators 
introduced a high-order randomness at the level of the convolution array size (9x9 pixels). 
This suggests that larger convolution arrays may provide more effective matching. 
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Figure 6: Evolved convolution and enhanced image with fitness = 2018851897. 

A low fitness result that provided better matching appears in Figure 6 above. This operator 
was generated in the 100th generation. It matches almost negatively with the template in 
Figure 4, yet it provides very good detail extraction and was in fact used in a Marr-Hildreth 
edge detector to find features in the sample frame (shown in Figure 4) and others produced in 
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time-sequence in the same manner as our example. Note that the fitness is "close" to half the 
maximum value of an unsigned 64-bit integer. 



Conclusions 

We have a number of conclusions and possible paths of continued research from these 

initial results. 

• Improved Fitness Evaluation. We need a fitness evaluation methodology more 
appropriate for the very large number of calculations required to compare these results 
pixel-by-pixel. One approach would average local results before doing the sum of squares. 

• Larger Convolution Kernels. Since we are working with images of 640x480 pixels, we 
have the potential to see better results with larger convolution kernels or using multiple 
kernels and linearly combining the resulting images. Plans are now under way to 100x100 
pixel kernels using a cluster computer. 

• The Utility of the FPGA vs. Clustered Systems. The FPGA is very useful for fitness 
evaluation and execution of small kernels because the convolution mathematics can be 
designed into the circuit. The XC6200 in particular is very useful when we convolve a 
frame using multiple kernels. For large convolution kernels, however, multiple FPGAs are 
needed. This is because the data bus of the FPGA becomes a limitation when evaluating 
large sets of numbers. We will be exploring the use of matrix math operations built into 
modem COTS microprocessors (e.g., MMX, 3Dnow) to optimize calculation time for 
matrix operations. We hope to produce a performance comparison evaluating COTS 
techniques for integer matrix calculations in the future. 

• Applicability to Motion Detection. For some very radimentary types of objects, our 
techniques could prove useful for identifying "custom" kernels that correspond to particular 
objects. For example, a robotic vehicle could use this method to isolate unrecognized 
objects and save a library of discovered templates for future reference. We have no 
conclusive results along this line, only the suggestion of value. 

• Overall Viability. The methodology described in this paper proved useful for our 
particular application. We hope to extend the methodology to provide useful generation of 
automatic convolution kernels for other general applications. 
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Abstract. In this paper we introduce a new approach to adaptive stereoscopic 
Vision. We use genetic programming, where the input to the individuals is raw 
pixel data from stereo image-pairs acquired by two CCD cameras. The output 
from the individuals is the disparity map, which is transformed to a 3D map of 
the captured scene using triangulation. The used genetic engine evolves ma- 
chine-coded individuals, and can thereby reach high Performance on weak com- 
puter architectures. The evolved individuals have an average disparity-error of 
1.5 pixels, which is equivalent to an uncertainty of about 10% of the true dis- 
tance. This work is motivated hy applications to the control of autonomous hu- 
manoid robots - The Humanoid Project at Chalmers. 



1 Introduction and Motivation 

Man is the Standard for almost all interactions in our world where most environments, 
tools and machines are adapted to the abilities, motion capabilities and geometry of 
humans. Walking robots have a very large potential in environments created for hu- 
mans as well as in more natural terrain. The largest potential is associated with robots 
of human-like dimensions walking on two legs - humanoid robots. It could be more 
efficient to control various machines by these robots than to rebuild all machines for 
direct Computer control [4] . 

This work is part of the Humanoid Project at Chalmers University of Technology. 
The project plans a series of humanoid experiments, all of which will be primarily 
controlled by evolutionary adaptive methods. 

The final goal of the research is to build a human-sized robot based on a plastic 
human Skeleton to ensure geometric authenticity. The current status of the project is 
that a second-generation prototype of a small humanoid is being developed - ELVIS - 
with a height of about 60 cm. 

Vision is the most important of our five senses. As an example a quarter of the hu- 
man cerebral cortex is devoted to sight', but some animals devote over half of their 



' According to a new estimate by neuroscientist David Van Essen of Washington University in 
St. Louis. 

S. Cagnoni et al. (Eds.): EvoWorkshops 2000, LNCS 1803, pp. 12-21, 2000. 
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brain capacity to the task of interpreting signals from their eyes (this even though their 
eyes has build-in object detection and as in some frogs initiates reflexes such as catch- 
the-fly - independently of their brains) 

There are two main reasons for using stereovision in robotics. It is a passive sensor 
without interference with the environment, nor with other sensor devices. It is also a 
reliable and effective way to extract range information, and it is easily integrated with 
other vision routines such as object recognition and tracking. For humanoid it can be 
motivated by its use in nature. 



2 Method and Experiments 

The genetic programming (GP) approach used in all experiments below is called 
Automatic Induction of Machine Code Genetic Programming (AIMGP) [5]. AIMGP 
is a linear GP-system that evolves binary machine code. The main benefit in this, 
compared to conventional systems, is a high increase in speed (about 60 times). The 
algorithm uses a steady state tournament selection, and the individuals are coded using 
a linear representation. The table below shows the most important settings that were 
used for all experiments. 



Table 1. The most important parameter settings 



Description 


Value 


Size of population 


15000 


No of demes 


5 


Size per deme 


3000 


Migration rate 


1% 


Crossover rate 


90% 


Mutation rate 


90% 


No of trainings 


150-200 


No of tests 


150 - 200 


Iterations 


40-200 


Constants 


0.5, 1.0,60.0 



Each selected individual was trained on 1 50 - 200 training examples (correspond- 
ing or interest point-pairs, see below). Each training example was iteratively run be- 
tween 40 and 200 times, see Fig. 3. Finally the best individual was tested on 150 - 200 
test examples. 

The input for the GP individuals consisted of stereoscopic image data and the inter- 
est point in question. An interest point is the position in one of the images (here the 
left) in a stereoscopic image-pair, for which a corresponding point in the other image 
(the right) is sought. 

The output is expected to be the corresponding interest point in the right image, 
thus giving information on the disparity of that point-pair. Some experiments expected 
the output to be the disparity value directly. Given the disparity and knowing the ge- 
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ometry of the set-up is enough to calculate the distance between the robots eyes (CCD 
cameras) and the real world position marked by the interest point-pair on the stereo- 
scopic image-pair. This is done using triangulation. 

The distributon of training examples 




Disparity in pixeis 



Fig. 1. One of the training sets 

The scene used to capture stereoscopic image-pairs was made up of the white in- 
side faces of a box, with the front side removed. One or more objects, placed in differ- 
ent positions in the scene, were used for each stereoscopic image-pair. 



Fig. 2. A stereoscopic image-pair with marked interest points 

Objects such as a black torch and a red floppy-disc box were used for the images in 
the first training, testing and validation sets. 

Two different camera models were tried out, the QuickCam™ VC and the Kodak 
DVC 323. A resolution of 160x120 pixels with 256 colours^ was used. 




^ 24-bit RGB values. 
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The training/evolution of the individuals in this first set of experiments was done 
off-line; i.e. on a stand alone PC instead of on the robot’s on-board subnotebook. The 
captured images were saved in a bitmapped file formal. Interest points in these im- 
ages were marked manually using a for-the-purpose-developed programme (see Fig. 
2), which saved the coordinates of the interest point-pairs in a data-file. Each image- 
pair had its own data-file, usually with between 10 and 40 interest points-pairs. 



2.1 Experiment series I - The Moving Points approach 

The task of the first experiment was to find for each point in the left image, the corre- 
sponding point in the right image - i.e. to solve the correspondence problem. Each 
individual had control over two “measuring points”, one in each image in the stereo 
image-pair, to be able to freely scan though the images in whatever way it found nec- 
essary. The area of an image available for an individual was restricted to a rectangle 
enclosing the interest point and its neighbourhood (a neighbourhood rectangle) - much 
the same as the sweep area window in Fig. 4. The individuals get the position of one 
interest point in the left image as input. By moving around the two measuring points in 
the two images, they search for the corresponding interest point in the right image. 




Fig. 3. An individual 

Positions in the image were given as linear^ coordinates inside the neighbourhood- 
rectangles. Origo were placed at the position of the left point (in the left image). The 
expected output was the disparity value for the two corresponding points, i.e. the hori- 
zontal distance between the two points. The early stage experiments expected a linear 
coordinate for the corresponding left point - the output - instead of the disparity. 

To be able to solve this task, each individual was repeatedly executed - between 60 
and 200 times. The individual got, at each iteration, the colour information of the two 



^ Microsoft Windows BMP, a device-independet bitmap (DIB) format. 
L = <width of img>*Y + X. 
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points defined in the previous iteration, the previous disparity estimation and the feed- 
back memories, with the hope that it step-by-step would improve the output, see Fig. 

3. 

2.2 Experiment series II - The Sweeping Box Approach 

The approach in this experiment was to find for each point in the left image, the corre- 
sponding point in the right image through scanning/sweeping. The disparity is then the 
distance (or the horizontal shift) between these two points. The neighbouring pixel 
data to an interest point (i.e. a sub-image) in the left image were fed as input to the 
individual, together with the same amount of pixel data around a sweeping point in the 
right image. Table 2 below shows that the sub-images consisted of 25 pixels, arranged 
in a 5x5 square. 



Table 2. The inputs and output for an individual 

INPUTS (v[]) OUTPUTS (f[]) 

0 24 left image data 0 matching score 

25 49 right image data 

The expected output from the individuals was a number that would describe how 
well the left sub-image matched the right sub-image. The individual was iteratively 
run for each pixel position in the sweep area window. See Fig. 4 for details. The high- 
est peak in the output of an individual was recorded, and the corresponding sweeping 
point location was taken as the individual’s answer of what should be used as data for 
the disparity calculation. All sweeps were parallel with the epipolar line, and the most 
common set-up was to do only one sweep per sweep area window. Two corresponding 
points should in theory be found on the epipolar line. This is not always the case, be- 
cause of several reasons: inexact camera mounting, minor sampling errors and non- 
successful timing synchronisation (when taking the stereo-image snapshots). The 
offset error is not usually more than one row of pixels. 

The closer an object were the cameras when the stereo-image pair was shot, the 
further apart the corresponding points (i.e. a higher disparity value). An object at infin- 
ity would have zero disparity. Due to this fact, the size (width) of the sweep area win- 
dow was set to 40 pixels. This corresponded to the minimum distance - and therefore 
maximum disparity - an object could have in the chosen scene set-up. 
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Fig. 4. The sweeping box approach. 

Descriptions of the functions and values of the variables and constants in the figure 
above can be found in Table 3 below. 

Table 3. Descriptions to Fig. 4. 

Name Description Value 

lx, ly Coord, of the left interest point, 

rx, ry Coord, of the right interest point, 

dx, dy Width and height of sub-image used as 5x5 pixels 

input to the individuals. 

Error Distance from the correct position. 

swp_dx, Width and height of the sweeping area 40x5 

swp_dy window. 

lox, loy Origo of the left box (sub-image), 

rox, roy Origo of the right sweeping box. 

sox, soy Origo of the sweeping area window. 

X coord An x-axis along swp_dx. 



0-40 
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y coord 
GP output 

x_resolution 

y_resolution 



An y-axis along swp_dy 

The matching score output from 

individual. 

Width of the stereoscopic images. 
Height of the stereoscopic images. 



0-5 



an 



160 pixels 
120 pixels 



3 Results 



Below follows the results of the two applied approaches. First the less successful ap- 
proach where the program is free to wander around in the pixels of the image and 
secondly the better working approach where a fixed scan path is used over the pixels 
in the image. 



3.1 Experiment! 

The evolution in this approach progressed slowly, and the majority of the individuals 
had an average disparity-error of around 14 pixels. That is about ten times more than 
is accepted to reach a useful accuracy in the distance estimation calculations. 

The main problem was that the evolution got stuck in local minima. A disparity of 
14 pixels is close to the average shift of the interest points in some of the training sets. 
Due to this fact, a majority of the individuals gave a constant output - a disparity of 14 
pixels. The lazy nature of an evolutionary process kept itself in this local minimum. A 
couple of minor variations of the algorithm were tried out, in the hope of finding a 
way to force the individuals to search for structures in the images, but they did not 
improved the results. Filtering the points in the training set (Fig. 1) in a way that 
caused two groups of points - one representing “close” objects and the other objects 
“far away” - did not force the individuals to find a strategy to classify them as "far 
away" or "close". 

A positive result from this experiment was that the genetic system evolved a 
sweeping behaviour for the control of its two freely movable points. But this did not 
help the individuals enough to find usable structures in the image data, retrieved from 
the position of the moving points. 



3.2 Experiment II 

This approach gave better results, and the evolution of useful individuals was really 
fast. The resulting average disparity-error of 1.5 pixels is acceptable and results in 
useful distances from the triangulation calculations. This disparity-error corresponds 
to an uncertainty of about 10% of the real world distance, which accurate enough for 
the first navigation experiments with our robot. 

Variations in the experiment were tried out, and the evolutionary process pro- 
gressed with the highest speed when giving the image-data in grayscales, instead of 
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RGB triplets. The conversion was done using the luminance/brightness parH in the 
YIQ colour model. 

The figures below show the recorded behaviour of three individuals. These particular 
individuals are evolved over 400 generations, but good individuals arise already in 
generation 60. However, some of these early individuals tend to make infrequent large 
mistakes (up to eight pixels). 

The blue graph (GP output - left axis) shows the individuals output value (their 
matching “probability”) that they produced during one single sweep. The yellow graph 
(Error - right axis) shows the disparity-error in pixels that the individual would pro- 
duce, if it used the current “Max output” (purple graph - left axis) as its final an- 
swer/output. The bottom scale shows the pixel-position of the sweeping box in the 
sweeping area window. 
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A recorded sweep (gen. 402) 
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Fig. 5. Staircase behavior 
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5 Y = 0.299*R + 0.587G + 0.144B. YIQ is used by the NTSC. 
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A recorded sweep (gen. 399) 




Fig. 6. Threshold behavior 
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Fig. 7. A 2-pixel matching error 



4 Future work 

Object recognition is our next goal. One way would be to evolve 3D scenes using 
simple geometrical objects, e.g. spheres, boxes and planes. The fitness function would 
describe how well the evolved 3D scene matched the 3D scene calculated from the 
disparity map produced by the approach discussed in this article. This is true in the 
real world - even though you only see the front side (a “square”) of a fridge, it is actu- 
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ally an object with a depth a “cube”. This method would give a way of generalising 
from “flat” images into hypothesises of a full 3-D model of the world. 

Future work on the sweeping box approach would be to include the shape of the 
sweeping box in the evolutionary process. 



5 Summary and Conclusions 

We have presented the first approach to stereovision using genetic programming and 
unprocessed pixel data. The resulting machine code individuals are efficient and en- 
able real time 3-D map generation. However, much research remains from this proof 
of concept experiment but several directions of the approach seem promising. We also 
regard evaluations in the autonomous robot as an important line of research for the 
future. 
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c . is pap r pr s ts a 1 g tic cl st ri g alg rit c 
i i g a g tic alg rit ( ) it t classical ard - a s cl s- 

t ri g alg rit ( ). It pr c ss s partiti atric s rat r t a 

sts fc trpitsadt spr idsa ipl tati sc 
f r t g tic p rat r - r c i ati . r c paris f p rf r a c 
it t r isti g cl st ri g alg rit s, a gra -1 1 i ag q a ti a- 

ti pr 1 is c sid r d. p ri tal r s Its s t at t pr p s d 
alg rit c rg s r q ickl t t gl al pti a d t s pr - 
id s a tt r a t f t dil a i ic t traditi al cl st ri g 
alg rit s ar asil trapp d i 1 cal pti a a d t g tic appr ac 
is ti c s i g. 



r c 

1 sterin ethods pla a ital role in e plorator data anal sis. In the eld 
of pattern reco nition [ ], the con entional - eans cl sterin al orith s ( 
s) ha e been idel applied, roadl spea in , s can be classi ed into 

r s (H ) and s { ) cl sterin al orith s. Hard 

cl sterin deals ith assi nin each object point to e actl one of the cl sters, 
hereas f cl sterin e tends this concept to associate each object point to 
each of the cl sters ith a eas reofbelon in ness, he belon in ness of object 
points to the hard/f cl sters is t picall represented b a e bership atri 
called a hard/f partition, respect! el .In eneral, s ai at ndin the 

opti al partition and opti i in a cl sterin object! e f nction sin calc 1 s- 

based ethods. Ho e er, cl sterin object! e f nctions are hi hi non-linear 

and Iti- odal f nctions. s a conseq ence, b hill-cli bin , s can be 

easil trapped into local e tre a associated ith a r partition (i.e., a 

partition ith one or ore e pt ro s, eanin that fe er than cl sters ere 

obtained in the nal partition), oreo er, the are also si ni cantl sensiti e to 
the initial conditions. 

a to achie e both a oidance of local e tre a and ini al sensiti it to 
initial! ation is to se stochastic opti i ation approaches, s ch as r 

r s ( s). n is inspired b or anic e ol tion and has been idel 

belie ed to be an effect! e lobal opti i ation al orith . In [3] [6], eneti- 

call ided approaches ere de ned for the opti al cl sterin proble s and 
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e peri ents ere perfor ed ith different data sets, es Its sho ed that an 

can a eliorate the diffic It of choosin an initiali ation for the s 

and pro ides a iable a to a oid local e tre a. Ho e er, as stated in [3], an 
cl sterin approach ta es p to t o orders of a nit de ore ti e than 
H / cl sterin al orith s. 

s are iterati e sche es operatin on a pop lation of so-called indi id - 
als. ach indi id al in the pop lation corresponds to a iable sol tion to the 
proble in hand. In pre io s or on eneticall ided cl sterin ([4] [6]), 

each indi id al in the pop lation has been desi ned to represent a atri of 

cl ster centers. Itho h nder t o sets of cl sterin criteria the appin be- 

t een a partition and the correspondin set of cl ster centers is e cl si e, the 

oal of a t pical cl sterin approach is essentiall the opti al partition rather 
than the positions of cl ster centers. In this paper, a no el eneticall ided 
cl sterin al orith is de ned, hich ses partition atrices as the indi id - 

als in pop lation. his h brid al orith co bines a enetic al orith ( ) 

ith the classical hard - cans cl sterin al orith (H ) and is therefore 

ter ed as a enetic hard - cans cl sterin al orith ( H )• H 

has been applied to a ra -le el i a e q anti ation proble . o pared ith 
H and approaches, H con er es faster and al a s to an 

nonde enerate partition hich is or is nearb the lobal opti 



r r 

onsider a set of ectors X = ... to be cl stered into ro ps 

of li e data, ach 3? is a feat re ector consistin of real- al ed ea- 
s re ents describin the feat res of the object represented b . Hard or f 
cl sters of the object points can be represented b a hard/f e bership 

atri called a hard/f partition, he set of all nonde enerate hard 

partition atrices is denoted b and de ned as 

= 3 ? 

; * ; ( ) 

here is a hard partition atri . he n ber of possible ’s, i.e. 

the n ber of a s of cl sterin objects into none pt ro ps, is a tirlin 

n ber of the second ind i en b (see [7]) 

(!)(-)- • ( 2 ) 

he cl sterin objecti e f nction for hard - eans (H ) partition is the 
H f nction 



( ) = 



{v ) 



( 3 ) 
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here = [?; v ... v ] is a atri of protot pe para eters (cl ster centers) 
V 3? ; and (v ) is the clidean distance bet een the th feat re 
ector and the ith cl ster protot pe v , hich is of the for ( ) = 

( — )( — ). his objecti e f notion describes the acc lated sq ared 
error hen replacin each feat re ector b the center of cl ster to hich it 
belon s. Hence, it is act all a eas re of distortion. 

ini i in the cl sterin objecti e f notion ith respect to leads to the 

folio in sets of cl sterin criteria, or each cl ster i {i [ ]), rstl , 



V = (4) 

i.e., the cl ster centers are positioned at the center of ass of the feat re ectors 
belon in to the cl ster. he second set of criteria that ini i es the H 
f notion states that an objecti e point sho Id be associated ith the closest 
cl ster center: 

^ if *= in [ (t’ )]; / N 

other ise. 

In eq ations ( ) { ),i = 2 . . . denotes the inde n ber of a cl ster or 

a cl ster center and = 2 . . . denotes the inde n ber of an object point. 

H adopts both sets of these cl sterin criteria and ini i es the objecti e 

f notion ( ) b alternati el pdatin and sin eq ations (4) and ( ). 

3 c s r g g r 

tartin ith an initial condition, a e ol es a pop lation to ards s cces- 

si el better re ions of the search space b eans of rando i ed processes of 

r , , and s . he i en opti i ation proble de nes 

an en iron ent that deli ersaq alit infor ation ( ss s)forne search 
points, and the selection fa ors those indi id als of hi her q alit to reprod ce 
ore often than orse indi id als. he reco bination echanis s alio s for 
i in of parental infor ation hile passin it to their offsprin , and tation 
introd ces inno ation into the pop lation. 

In brief, a search for the opti al indi id al is t picall i pie ented as 
folio s. 

enerate an initial pop lation; 

2. al ate the tness al e of each indi id al in the c rrent pop lation; 

3. elect pairs of parents; 

4. enerate offsprin of the selected parents ia reco bination and tation; 
eplace the parents ith their offsprin and create a ne eneration; 

6. Halt the process if a ter ination criterion is et. ther ise, proceed to step 

3. 

o red ce the search space drasticall , e introd ce H into the t pical 

i pie entation process. rin e er eneration, ith tness e al ation 
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ta in place, a sin le H reallocation step is applied to each indi id al in 

the pop lation. ccordin to the partition atri represented b an indi id al, 
the set of cl ster centers are located sin eq ation (4) and then the partition 
atri is pdated b eq ation ( ),i.e. b assi nin each object point to its closest 
cl ster. fter all the indi id al partitions are pdated sin H , the three 

basic enetic operations - selection, reco bination and tation - start. 

In order to appl a enetic approach to a i en proble , a n ber of f nda- 
ental iss es st be addressed in ad ance. he are sol tion representation, 
tness f notion, creation of the initial pop lation and a s cceedin ne ener- 
ation, i pie entation sche es of enetic operators, ter ination criterion, and 
the para eter settin s. he rest of this section describes each of these iss es 
in detail. 



. t pr s t t 

s entioned in section , each indi id al in the pop lation is a hard partition 
atri . In hard cl sterin , an object point belon s to the closest 

cl ster e cl si el . here is onl a do n an col n of a hard partition atri . 
Hereof, it is possible to si pif a hard partition atri into an - 

di ensional ector u ith the ith ele ent describes hich ro the lies do n 
the ith col n of the ori inal . he possible al es of the ele ents of u ran e 

fro to . his si pli cation is adopted in the proposed H 

.2 t ss ct 

o alio CO parison of perfor ance ith the classical H as ell as an 

e istin enetic cl sterin approach- [3]), H f nction ( ) is sed 

here as the objecti e f nction to be ini i ed. nd the atri is calc lated 

ith respect to sin eq ation (4) . o lar el red ce the chance of H 
beco in st c at a de enerate partition, e ha e ta en the n ber of e pt 
cl sters into consideration, n al e obtained sin H f nction is scaled 
ith a penalt factor, ifferent fro the one in [3], o r objecti e f nction is 
rede ned as folio s: 

()=()(+) ( 6 ) 

here is the total n ber of cl sters and denotes the n ber of 

e pt cl sters and is e al ated ia co ntin the all- ro s in 

he oal of a cl sterin approach is to ini i e the objecti e f nction hile a 
fa ors tter indi id als. o co pro ise, e se the in erse of an indi id al’s 
scaled H f nction al e as its tness al e. In addition, a linear tness scalin 
echanis [ ] has been introd ced to aintain reasonable selection press re. 

t t 

onsider that ood choice for startin con rations sho Id be free of o ert 

biases, or the hard partition ectors u in the initial pop lation, each ele ent 
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i . . rap ical d script! ft r c i ati strat g it is appli d t t 

t cl st r f par t_ ’s partiti u . 



is set to a rando 1 enerated n ber in the ran e of [ ]. doin so, e 

act all rando 1 partition the object points to initial cl sters. 



.4 t c p r t rs 

In e er eneration, a select parents fro the c rrent pop lation. heo- 
reticall , the probabilit of one indi id al bein selected is proportional to its 
tness al e relati e to the others’ tness al es. fter bein selected, parents 
are ated to i e birth to their offsprin . ffsprin are enerated ia the oper- 
ations of reco bination and tation. 

. ct . s to the selection operator, the s s rs s 

sche e is applied [9]. ased on the theoretical and e pirical anal sis, a er 
concl ded [9] that this sche e is an opti al seq ential sa plin sche e hich, 
for the rst ti e, assi n offsprin accordin to the theoretical sped cations. 

c t . brand-ne reco bination strate has been desi ned 

for the cl sterin proble s. It reallocates the orst e ber in each cl ster of a 
parent’s partition accordin to its atin partner’s partition, he orst e ber 
of a certain cl ster is the farthest object point to the center of that cl ster, a on 
those belon in to it. 

artic lari , for the ith cl ster of parent ’s partition u , reco bination is 
carried o t as folio s: 
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. nd the orst e ber of the ith cl ster of u (s ppose it’s ); 

2. accordin to the atin partner’s partition ector u , nd o t to hich 

cl ster this orst e ber belon s (s ppose it’s the th cl ster of tt ); 

3. still accordin to u , rando 1 choose another object point fro the th 
cl ster of u (s ppose it happens to be ); 

4. bac to u , chec o t to hich cl ster of u the chosen object point 

belon s and si pi reallocate to that cl ster. 

hile still holdin the rando ness propert , this reco bination is ell ided. 
In step 2, the atin partner’s partition u is referred to ans er the folio in 
q estion: hich object points sho Id share the sa e cl ster ith the orst e - 

ber of a certain cl ster of partition u ? ince there is at least one (itself) hile 
a be ore than one object points a ailable, step 3 rando 1 choose one of 

the . inall , in step 4, the orst e ber of partition u ’s ith cl ster is reallo- 

cated to a ne cl ster s ch that it belon s to the sa e cl ster as this rando 1 
chosen object point, or better nderstandin , these i pie entation steps are 
described raphicall in i re . 

he reco bination process is applied to each parent of the atin pair cl ster 
b cl ster. hat is to sa , each cl ster is adj sted independent! of the others. 

c. t t . fter e er reco bination, tation is i posed on each ele- 

ent of the ne 1 constr cted partition ith a tation probabilit p . ta- 

tion sets the chosen ele ents to a rando 1 enerated inte er ran in fro to 



rtf r t 

r enetic cl sterin al orith is a stead -state , hich replaces onl a 
fraction of the pop lation each eneration. he oti ation of introd cin a 
stead -state is to eep a ood balance bet een e ploitation of the best 
re ions fo nd so far and contin ed e ploration for potentiall better pa off area. 

Ho e er, a stead -state ill increase the ariancealon the ro the r es 
of indi id als [ ] . o red ce the ariance, a irst-In- irst- t ( I ) deletion 
[ ] is e plo ed and th s per it the se of this stead -state ith s aller 
pop lations. ith I deletion the pop lation is si pi a rst-in- rst-o t 
q e e ith ne indi id als added to one end and deleted indi id als re o ed 
fro the other end. 

oreo er, elitis is i pie ented to eep the best e ber of the pop lation. 

.6 r t r tt gs 

he enetic cl sterin approach is co p tational e pensi e. he do inant cost 
is that of calc latin the tness al e for each indi id al in the pop lation for 
e er eneration. his cost is a f notion of , , and , i.e., the n ber of 
feat res, n ber of object points, and n ber of classes to cl ster the object 
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points, n increase in an of these para eters res Its in a linear increase in ti e 
per eneration. 

ince the total co p tational ti e is also proportional to the pop lation si e, 
s all pop lations are preferred, hro ho t the e peri ents ith H , 

e sed a pop lation si e of 3 . 

In e er eneration, 6 % indi id als of the pop lation ill nder o the three 
basic enetic operations to enerate offsprin . he probabilities of s cceedin re- 
co bination and tation are .9 and . , respecti el [ ]. hese para eter 

settin s ha e been fo nd to offer best res Its. 

he ter ination criterion sed here is the n ber of enerations. he ap- 
proach stops hen the req ired eneration is created. 

p r s 

s cl sterin is often applied to i a e processin and i a es are real- orld 
do ains of si ni cant co pie it in ter s of n ber of object points to be 
cl stered and n ber of classes, a ra le el i a e q anti ation proble is 
considered. In this application case, the 2 6 2 6 blac -and- hite ena i a e 

is rstl di ided e enl into s all bloc s of 4 4 pi els. hen the ra le els 

of the pi els in each of these bloc co pose a ector s ch that there are 4 96 
i a e ectors of 6 feat res ( ra le els), he oal of this i a e q anti ation 
proble is to cl ster these 4 96 i a e ectors into 2 6 classes. Hence, = 4 96, 
= 2 6, and = 6. 

o this i a e q anti ation proble , the classical H , a present enet- 

icall ided cl sterin al orith - [3], and o r o n H ha e been 

applied independent! . nd the object points processed b all of these al orith s 
ere the 4 96 i a e ectors consistin of 6 pi el ra le els. 

he classical H as i pie ented as folio s: 

ando 1 partition the i a e ectors into initial cl sters. ale late the 

correspondin set of cl ster centers sin eq ation (4) . 

2. q ation ( ) is applied s ch that accordin to its 6 pi el ra le els, each 

i a e ector is assi ned to the closest cl ster center. ean hile, the 

clidean distance bet een each i a e ector and its closest cl ster center is 

s ed to the acc lated sq ared error 

3. q ation (4) is applied, ach cl ster center is pdated as the center of ass 
of all the i a e ectors belon in to it. 

teps 2 and 3 are applied alternati el ntil the relati e difference in the 
acc lated sq ared error of t o s ccessi e iterations is less than = . 

(i.e. abs( _ — _ rr ) _ rr )• tep 2 is repeated sin the last set 

of cl ster centers, he res Itin acc lated sq ared error and n ber of 
e pt cl sters are recorded. 

Hall . desi ned the approach folio in the t pical i pie entation 

steps of a and the H as also introd ced in each eneration. he 

three enetic operators ere to rna ent selection, t o-point reco bination and 
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rando tation. In , indi id als of the pop lations ere sets of cl ster 
centers instead of partition atrices. 

or CO parison, the approach has been repeated. Ho e er, there are 

so e differences bet een ore peri ents and theirs and these are hi hli hted 
as folio s: 

hile the had sed -fold to rna ent selection ith =2, e ha e sed 
stochastic ni ersal sa plin sche e, as in o r o n H H 

2. hile the had sed a enerational ith an elitist strate of passin 

the t o ttest indi id als to the ne t eneration, e ha e sed a stead - 
state ith eneration ap of .6 thro ho t. litis is i pie ented to 

eep the ttest indi id al. 

3. hile the had sed a binar ra code representation for the indi id als, 
e ha e sed real al e representation. 

i e those sed thro ho t the e peri ents for o r enetic cl sterin al o- 
rith , e ha e chosen the sa e para eter settin s in all the repeated 
approaches, he ere a pop lation si e of 3 , a reco bination probabilit of 
.9 , and a tation probabilit of . . cept that better sol tion can be 

obtained ith lar er pop lation si e, these al e offer the best perfor- 

ance [3]. er approach stopped at the 4 th eneration. 

S S 

ests on cl sterin i a e ena b the classical H , the approach, 

and or H are nderta en respect! el . he e eas res for co parin 

al orith perfor ances are the ean sq ared error ( ) and n ber of e pt 

cl sters, , associated ith a partition atri . he ean sq ared error is the 
al e of the H f notion a era ed b the n ber of i a e pi els ( = 

(2 6 2 6)), hich indicates the distortion bet een the q anti ed i a e and 
the ori inal i a e. rin each r n, the partition ith the lo est acc lated 
sq ared error is traced and the and al es associated ith the opti al 
partition nail fo nd are recorded. 

he recorded al es of and are a era ed o er the total n ber of r ns, 
i.e. , 4 and 4 for H , and H respect! el . he a era e 

al es and the standard de iations of and are reported in able . he 
and al es associated ith the er best partition e er obtained b each 
al orith are also incl ded. 

i re 2 sho s the histo ra of the recorded and al es for H , 

and H r ns, respect! el . he histo ra is a statistical represen- 

tation of the distrib tion of local opti al obtained b the al orith . ddin 
res Its of ore r ns did not chan e the distrib tion f rther. 

ro able and i re 2, the folio in obser ations are dra n: 

. Usin H , a broad distrib tion of local opti al is obser ed. he stan- 

dard de iations of and are ch hi her than those of the enetic 
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approaches, his indicates that classical H is sensiti e to the initial 

distrib tion of cl ster centers and easil trapped in local opti a, hile, on 
the other hand, sho s the iabilit of a enetic approach to o erco e these 
proble s. 

2. oth enetic ided cl sterin al orith s o tperfor the classical H 

in the sense that the end p in partitions associated ith ch lo er distor- 
tion and si ni cant fe er e pt cl sters. artitions of si ilar al es 

ere repeatedl fo nd b and H . his indicates that these 

res Its are indeed nearl opti al. 

3. In no case did H res It in a de enerate partition, n a era e, the 

best al e obtained b H is sli htl lo er than that obtained 

b . o is for the er best e er fo nd. 

rin each eneticall ided cl sterin approach sin either or 

H , the al e and e pt cl ster n ber of the best partition 

fo nd p to and incl din each s ccessi e eneration are recorded and then 

a era ed o er the total n ber of r ns. i re 3 sho s the res Itant a era e 

al es of and ith respect to the eneration n ber for both and 

H . ccordin to the to c r es, e see that, for both enetic 

approaches, the initial con er ence rates are er hi h and as the enerations 
pro ress con er ence rate decreases rapidl . Ho e er, in the earlier enerations, 
H con er es ch faster than and q ic 1 reach the desired re- 
ion here the nonde enerate partition atrices reside. H can nd an 

al e lo er than the lo est e er fo nd b H after the th eneration, 

hile does it after eneration . s stated, for cl sterin proble s the t- 

ness e al ation at e er eneration is ti e cons in . Itho h soon 

catches p and fro eneration the difference in al es bet een t o 

enetic cl sterin al orith s is o ittable, in the special cases here the speed 
as ell as perfor ance is req ired H a pro ide a ch faster a 

to nd an acceptable sol tion. rther ore, as H ass res local opti al- 

it and d e to its hill-cli bin ethod con er es ch faster than an enetic 
approach, instead of aitin for the enetic approaches reach an e act opti al 
sol tion, e a stop enetic search after a necessar n ber of enerations 
and se H to nd the correspondin local opti 
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6 c s 

In this paper a no el enetic cl sterin al orith is proposed, hich co bines 
a enetic al orith ( ) ith the classical hard - cans cl sterin al orith 

(H ). Unli e other cl sterin al orith s, H processes partition 

atrices rather than sets of center points and th s alio s a ne i pie entation 
sche e for the enetic operator - reco bination, or co parison of perfor ance 

ith other present cl sterin al orith s, e peri ents on a ra -le el i a e 
q anti ation proble ha e been cond cted. he res Its sho that H 

con er es ch q ic er to the lobal opti and pro ides a iable a to 

sol e the dile a here the classical H is fo nd easil ca ht in local 

opti a and a enetic approach req ires lar e ti e cons ption. 
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Abstract. The fitness function used in a GA must be measurable over 
the representation of the solution by means of a computable function. 
Often, the fitness is an estimation of the nearness to an ideal solution or 
the distance from a default solution. In image scene interpretation, the 
solution takes the form of a set of labels corresponding to the components 
of an image and its fitness is difficult to conceptualize in terms of distance 
from a default or nearness to an ideal. Here we describe a model in 
which a semantic net is used to capture the salient properties of an ideal 
labeling. Instantiating the nodes of the semantic net with the labels from 
a candidate solution (a chromosome) provides a basis for estimating a 
logical distance from a norm. This domain-independent model can be 
applied to a broad range of scene-based image analysis tasks. 



1 Introduction 

We describe how a genetic algorithm (GA) can be employed to classify /label 
objects in a scene for which no prior truth data exists. Relationships among the 
objects in a typical scene from the domain of discourse are encapsulated within a 
semantic net. The method was validated using a test suite of images captured by 
satellites. Mainly, these included the infrared band of North Atlantic scenes and 
two bands of AVHRR data depicting regions of the Western U.S. The objective 
of the North Atlantic image analysis was to identify currents such as the Gulf 
Stream and eddies. The objective of using AVHRR images was to detect and 
identify clouds by type. 

2 Background 

Classification or labeling of segments is the focus of this paper. Labeling a seg- 
ment of an image is a particularly difficult subtask because there must be an 
automatic method of assigning a figure of merit to a candidate solution. A very 

* This work was supported in part by a grant from NASA/Goddard Space Flight 
Center, :j5fNAG5-8570 and in part by DoD EPSCoR and the State of Louisiana 
under grant F49620-98-1-0351. 
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general and computationally reasonable method for defining fitness function for 
image labeling is described that is based upon developing a semantic net for a 
typical scene from the domain. The classes of the semantic net are defined by the 
labels to be assigned to the segments. The predicates are relationships that exist 
among objects bearing the corresponding labels. Given a candidate solution (a 
labeling) for an image, a measure is described that estimates the conformance 
of the labeling to the semantic net. 



3 Methodology 

In this section, the formal approach and extensions to our prior ad hoc model 
using semantic nets. [1] is developed. A semantic network is a structure used to 
represent knowledge as a combination of nodes interconnected with arcs. 

3.1 Description of Scene Properties with Semantic Nets 

Let CL be the set of possible classification labels (or classes) specific to the 
application. 

CL = {ci,C2, ■ ■ ■ ,Cm} (1) 

Let S be the set of segments/components in the given image: 

S' = {S1,S2, ■ ■ ■ ,Sn} (2) 

Each of these components is assumed to be completely characterized by T, the 
set of features specific to the application. Let, 

T = ■ ■ ■ , th} (3) 

For any given image component, each of these features, f,, takes values from 
some corresponding domain T>,. 

Assume that every statement about such application-specific knowledge that 
is useful for the classification task can be described by using at most kmax number 
of components drawn from the image. First, consider only those statements which 
involve exactly k (for some 1 < A: < kmax) components of the image. Let Ck be 
the set of all A:-tuples built from the indices of the elements of CL. i.e., 

Ck={{h,---,3k) I (4) 

Every image component s £ S can be an instance of only one of the m classes 
drawn from the set CL. Denote this by ISA, an instance function which maps 
each image component to the index of a class. 

ISA: S ^ {l,...,m} (5) 

Thus, given a A:-tuple of image components/segments, ISA can be used to gen- 
erate a corresponding fc-tuple (ji , . . . ,jk) & Ck. Let Fk be a set of feature-value 
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comparator functions such that each function f G maps a A:-tuple of feature 
values drawn from the domain corresponding to some feature € T, to an 
absolute comparison value. 

Fk = {f\f:Dl^n} (6) 

Let Pk be the set of predicates such that each p £ is & fuzzy predicate mapping 
a given absolute comparison value to a fuzzy truth- value in the continuous range 
of [0,1], 

Pk = {p \ p-.n^[o,i]} ( 7 ) 

Predicates based on fuzzy logic are more appropriate than binary predicates for 
this task in view of the heuristic knowledge employed. In addition, null £ Pk, 
and null(x) = 0 for all x. 

For every A:-tuple of image components, every feature-value based relation- 
ship (inherited from the corresponding fc-tuple of classes G Cu) 

among its elements can be quantified by using some specific feature-value com- 
parator function and a corresponding predicate. For a given fc-tuple of classes, 
(ill ■ ■ ■ ,jk) G Ck, there may exist multiple relationships among the elements of 
the A:-tuple of class-instances. Therefore, we define a mapping, Rlk such that 
every given tuple of classes denotes a specific subset of the available predicates, 
i.e., 

Rlk : Ck 2^“ (8) 

We also define another functional mapping R2k such that every predicate is 
mapped to a feature comparator function, i.e., 

R2k ■ Pk — > Fk (9) 

Thus, every relationship involving k components can be uniquely denoted by the 
tuple {{ji,. . . ,jk),p) where, (ji,...,jk) G Ck and p G Rlk{{ji, . . . ,jk))- Con- 
sider a A:-tuple of image components/segments that is instantiated (by virtue of 
some ISA mapping) to be a tuple of class-instances corresponding to (ji , . . . , jk). 
Every relationship {{ji, . . . ,jk),p) inherited by the image components can be 
quantified as follows: Suppose that / = R2k{p) and that / makes use of the fea- 
ture ti £ T using fc-tuple of values from the domain T>,. Now, this relationship 
can be quantified by the composition p(f(xi ,... ,Xk)) where x\, . . . ,Xk & Di are 
the feature values of the image segments. 

The set of all such relationships constitutes a semantic net, SNf.. In other 
words, 

SNk={{{h,---Ak),p) I {ji,---,jk) &CkAp£ Rlkiiji,- ■ ■ ,jk)) } (10) 

This formal model for representing and utilizing the knowledge relating k ele- 
ments can be summarized by FNk, a fitness net. 

FNk = {CL,S,T,ISA,Ck,Pk,Rlk,Fk,R2k,SNk,Wk,E) (11) 

where, Wk is a set of weights, one for each predicate p £ Pk, and E is a fitness 
function described in section 3.3. 
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This model can now be generalized to represent all possible statements from 
the knowledge base, i.e., for all values of A:, 1 < A: < kmax- Such a complete 
model is given by: 

FN = {CL, S, T, ISA, C, V, Rl,T, R2, SAf, W, E) (12) 

where, 



c = 


Ui<A:<A;™„,b 


V = 


Ui<A;<A!™„,b 


R1 = 




T = 


^KkKkmai 


R2 = 




SN = 




W = 









To illustrate the notions discussed above, consider a simple domain such as a 
chair shown in Figure 1. Here we have, CL = {s, b, a, 1}, where the symbols de- 




Fig. 1. A chair and its segments 



note, and index, the classes “Seat” , “Back” , “Arm” , and “Leg” respectively. Also, 
S = {si, s2, s3, s4, s5, s6, s7, s8) from Figure 1. There are two measures/features 
for each segment: angle of rotation from the horizontal, t\, and surface area, t 2 - 
While relationships of any degree (except zero) are permitted, for the sake of 
simplicity we assume only binary relationships for this example, i.e., we consider 
only the case of A: = 2 and C = C^. The set of feature comparator functions is 
given hy R = F 2 = {/i,/ 2 }- /i takes the angle-measures of two segments as 
arguments and computes their relative orientation, computes the difference 
in the surface area of the given argument pair of segments. Further, let the set 
of predicates be P = P 2 = {perpendicular -to, parallel-to, area-greater-than,null}. 
The predicates perpendicular-to and parallel-to return 1 if the segments are mu- 
tually perpendicular and parallel respectively. Otherwise, they return 0. Simi- 
larly, the predicate area- greater- than returns 1, if the difference computed by /2 
is positive and 0 otherwise. Obviously, fuzzy predicates can be used instead to 
assign values in the range [0,1]. The mapping from class tuples to predicates, 
TZl = RI 2 , is simplified in this case since no class pair has more than one rela- 
tionship. This is graphically shown in the top-half of Figure 2. The functional 
mapping, TZ2 = R22, can be constructed easily since predicates perpendicular-to 
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and parallel-to use the function /i and the predicate area- greater -than uses the 
function In this simple example, the semantic net SM = SN^ closely corre- 




Legend 

I : perpendicular-to 
H : parallel-to 
(X> : area- greater- than 



Fig. 2. A sample semantic net and a candidate ISA mapping for a scene consisting of 
a chair. 



spends to RI 2 and the Figure 2 shows the useful parts of the semantic net for a 
typical chair using the three predicates. Figure 2 also shows a particular instan- 
tiation, i.e., an ISA mapping, of the segments from the scene shown in Figure 1. 
This mapping of {(sl,s), (s2,a), (s3,l), (s4,b), (s5,l), (s6,l), (s7,a), (s8,l)} results 
in the correct classification of the segments. Also, it is obvious that any other as- 
signment of labels to the segments would result in a lower consistency evaluation. 
Formally, the semantic net shown in Figure 2 is given by 

iSA/" = { ((6, a), Q!>), {{b,l),null), {{b,s),±), {{b,b) ,null) , {{s,a),a>), 
((5,L),T), {{s,s),null), {{1,1), U), {{a, a) , null) } 

where, the symbols “q!>”, “T”, and “/ /” denote the predicates area-greater-than, 
perpendieular-to, and parallel-to respectively. 



3.2 Representing Candidate Solntions in GAs 

A candidate solution for the classification task takes the form of a vector of 
indices (ji, . . . , j,, . . . , j„), containing one element for each segment s, € S. This 
vector of indices represents a possible ISA mapping for the segments in the set S. 
In other words, ISA{si) = ji where, 1 <i <n and 1 < j* < m. For example, the 
vector (s. A, L, B, L, L, A, l) represents the labeling shown in Figure 2. Similarly, 
the vector (l. A, S, B, A, L, L, l) represents another candidate solution, albeit of 
inferior quality. 
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3.3 Computing Fitness Using Semantic Net Description 

Fitness is a quantitative measure of the consistency of an ISA relationship that a 
candidate solution represents. The procedure described below may appear to be 
computationally expensive. However, we have found that, in practice, there are 
very few relationships involving more than two components/segments and that a 
large number of high-order relationships are reduced to null predicates. Further, 
all the predicates can be precomputed for the segments in a given image and 
the repeated fitness computations need only do the summation of the various 
predicate values using a table look-up. 

kma^ 

^=E E E- X p{f{xi,. . . ,Xk)) (13) 

{{ck),p)eSN„ (sk) 

where {ck) abbreviates {ji, . . . ,jk) G Cu and {su) represents any fc-tuple of image 
segments instantiated to (ji, . . . ,jk) via the ISA mapping. Also, / = R2f.{p), 
w G Wk is the weight corresponding to predicate p, and x\,. . . ,xt G D, are 
the feature values of the image segments in (st) corresponding to some feature 
ti £ T, depending on the feature comparator function /. For domains in which 
knowledge is unevenly distributed in the semantic net, practice may dictate that 
normalization over the set of predicates for each class or class-tuple be performed. 

To illustrate fitness, examine the best solution (s. A, L, B, L, L, A, l) taken from 
Figure 2. Since there are four and two instances of the classes I and a respec- 
tively, we need to sum twenty-three predicate values, ignoring the null predicate. 
Assuming unit weights for all predicates, on summation, the fitness value for the 
ideal solution can be seen to be twenty three, since all the predicates have a 
value of 1 in this case. In contrast, consider the obviously sub-optimal solution 
(l, a, S, b, a, l, l, l). Here, only ten out of the twenty three predicate values have 
a value of 1 resulting in a fitness value of ten. 

4 Experiments and Analysis 

The goal in the oceanographic problem is to label the mesoscale features of the 
North Atlantic from satellite images using a known set of classes. The classes 
in this case are “Gulf Stream North Wall”(N), “Gulf Stream South Wall”(s), 
“Warm Eddy”(w), “Cold Eddy”(c), and “Other” (o), i.e., CL = {n, S,W, C,o}. 
We use edge-segmented images of the region in the infrared band (10.3-11.3 
pm) . An infrared satellite image and its companion segmented image are shown 
in Eigures 3 and 4. Eor the image shown, we have, S = {si, s2, . . . , s35). We 
use only two measures for each segment in the image - its position and length. 
They are computed by using coordinates of the centroid (based on the mass of 
the segment) and the two end-points. 

The set of predicates V and the set of functions T are informally shown in 
Table 1. The semantic net SM for the domain is shown in Table 2. The mappings 
III and 112 are implicit in these tables. We use unit weights for all predicates. 
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Fig. 3. Original Infrared Image of the Gulf Stream 
Table 1. Description of predicates and functions for oceanic labeling 



Predicate 


Function 


Comments 


Is-North-Of(i,j) 


If AvgLat{i) > AvgLat{j), 
= 1; Otherwise, = 0. 


Segment i is north of segment j 


Is-Near(i,j) 


exp(-/3 X Xij) 


Xij is distance between 
segments i and j. 


Is-Not-Near(i,j) 


1 — Is-Near(i, j) 


Fuzzy complement of Is-Near. 


Is-North- 0/- And 
-Fifty-Km-From(i,j) 


min{ Is-North- Of{i,j), 
exp(-/3 X \Xij - 50|) } 


Segment i is north of segment j 
and is 50 km from it. 


Arcs-Of- Circle-And 
-Less- Than-Hundred\ 
-Km-Distant(i,j) 


min{ Arcs-Of-Circle{i, j), 
Less- Than-Hundred- 
Km-Distant{i, j) } 


Arcs- Of- Circle is estimated based on 
intersection of cords from segments. 
Second predicate is computed as 
= 1, if Xij < 100, and 
= exp(— /I X \Xij — 100|), otherwise 



Also, it is noted that the predicates are computed a priori for all the segment 
pairs and stored in a lookup table. 

Candidate solutions are represented as vectors of labels. For the image shown 
in Figure 4, any label vector (ci , C2 , . . . , C35) such that c, € {n, s,w, c,o} for all 
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Table 2. A semantic net for oceanic segment labeling 



Class tuple 


Predicate Name 


{w, w) 


Is-Near(i,j) 


(w, w) 


Arcs- Of- Circle- And 
-Less-Than-Hundred-Km-Distant(i,j) 


{w,n) 


Is-North-Of(i,j) 


{w,n) 


Is-Near(i,j) 


{n,n) 


Is-Near(i,j) 


{n,s) 


Is-North-Of-And-Fifty-Km-From(i,j) 


(s,s> 


Is-Near(i,j) 


(s,c) 


Is-North-Of(i,j) 


(s,c) 


Is-Near(i,j) 


(c,c) 


Is-Near(i,j) 


(c,c) 


Arcs- Of- Circle- And 
-Less-Than-Hundred-Km-Distant(i,j) 


(o,w) 


Is-Not-Near(i,j) 


(o, n) 


Is-Not-Near(i,j) 


(o,s) 


Is-Not-Near(i,j) 


(o,c) 


Is-Not-Near(i,j) 


(o,o) 


Is-Near(i,j) 


Other tuples 


null 


Legend: 

W = Warm Eddy; C = Cold Eddy; 0 = Other 
N = North Wall of Gulf Stream 
S = South Wall of Gulf Stream 



i, constitutes a feasible candidate solution. Such label vectors are encoded as 
bit strings suitable for GA search. Table 3 shows the parameters used for the 
GA runs. Each run with these settings was repeated 10 times, each starting 
with a different initial random population. The accuracy of the best solution 
generated by the GA in each run with respect to the fairly difficult image shown 
in Figures 3 and 4 is compared with that of manual labeling and is listed in 
Table 4. Figure 4 shows the best labeling obtained over all the runs. 



5 Conclusions 

Here we describe a domain-independent framework for labeling image segments 
for scene interpretation. This approach is based on the abstract representation 
of a typical scene from the domain of discourse. The abstraction form, «.e., 
semantic network, permits encoding the descriptions of relationships of arbitrary 
degree among the instances of scene objects. A GA is used in searching the space 
of candidate solutions for the best labeling. Fitness of a candidate solution is 
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Table 3. Parameters of GA runs for oceanic labeling 



Description 


Value 


Population size 


200 


Number of generations 


200 


Selection operator 


Proportional selection using stochastic 
remainder sampling with replacement 


Crossover operator 


Uniform crossover (allele level) 


Probability of crossover 


0.600 


Mutation operator 


Bit mutation 


Probability of mutation 


0.005 



Table 4. Accuracy of GA-generated oceanic labeling 



Run ff: 


Accuracy % 


Run ff: 


Accuracy % 


1 


80 


6 


63 


2 


57 


7 


71 


3 


66 


8 


77 


4 


83 


9 


71 


5 


83 


10 


69 



estimated by evaluating the conformance of the solution to the relationships 
depicted in the semantic net. 
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Abstract. We introduce a test environment based on the optimization 
of signals approximated in function spaces in order to compare the perfor- 
mance of different evolutionary algorithms. An evolutionary algorithm to 
optimize signal representations by adaptively choosing a basis depending 
on the signal is presented. We show how evolutionary algorithms can be 
exploited to search larger waveform dictionaries for best basis selection 
than those considered in current standard approaches. 



1 Introduction 

In order to facilitate an empirical comparison of the performance of different 
evolutionary algorithms a test environment must be provided. Traditionally, sets 
of test functions with specific topological properties, commonly known as fitness 
landscapes, have been proposed by several authors to be used in performance 
benchmarking. In particular, the De Jong’s test function set has been a standard 
for genetic algorithm benchmarks since 1975. In most cases, the optimization 
objective is formulated as a global function minimization problem. 

In this paper, we depart from this view by considering the optimization of 
functions approximated in function spaces. Series expansions of continuous-time 
signals go back at least to Fourier’s original expansion of periodic functions. A 
basis is a set of linearly independent functions that can be used to produce all 
admissible functions f{t). The idea of representing a signal as a sum of elemen- 
tary basis functions, or equivalently to find orthogonal bases for certain function 
spaces, is very powerful. However, classic approaches have limitations, in partic- 
ular there are no ’’good” local Fourier series that have both time and frequency 
localization. An alternative is the construction of wavelet bases, which use scal- 
ing instead of modulation in order to obtain an orthonormal basis for (K) . An 
entropy-based algorithm for best basis selection has been proposed in the litera- 
ture [6]. Under the specific conditions of its application, the standard best basis 
(SBB) algorithm finds the optimum basis decomposition according to a spec- 
ified cost functional. We show that this algorithm can be used to benchmark 
evolutionary algorithms. 

A second objective of this paper, is to show how evolutionary algorithms 
can be exploited to search larger waveform dictionaries for best basis selection 
than those considered in current standard approaches. We extend the scope of 
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the SBB algorithm by searching larger waveform dictionaries in order to find 
better adaptive signal representations. In Sect. 3, we present an evolutionary 
algorithm for best basis selection. Adapted waveform analysis uses libraries of 
orthonormal basis and an efficient functional to match a basis to a given signal 
or family of signals. Two often used libraries are wavelet-packets and localized 
trigonometric functions, since they support the expansion of the waveforms in 
orthonormal basis whose elements have good time-frequency localization prop- 
erties. These libraries constitute huge collections of basis from which we can 
pick and choose the best matching basis. Flexible decompositions are important 
for representing time-frequency atoms whose time-frequency localizations vary 
widely. In this article, we propose the use of evolutionary algorithms [10] as the 
main searching tool for best basis selection. The proposed approach generates 
a population of solutions based on basis expansions of multi-filter, time-shifted, 
wavelet-packet libraries. An evolutionary algorithm operates on the population 
to evolve the best solution according to a given objective function. Libraries of 
bases represent the population from which we want to select the best-fit individ- 
uals. This optimization approach permits more flexibility in searching for best 
basis representations than traditional approaches. 



2 Wavelet Spaces 

In this section, we briefly review the framework of wavelet basis selection. Let x € 
K" be an input signal and let f2o,o = K" represent the signal space and Bo,o = 
(ei , . . . , e„) be the standard basis of K" . Wavelet packets split this original space 
into two mutually orthogonal subspaces smoothly and recursively, i.e., 

^j,k = ^j+l,2k ffi ^ij+l,2k+l (1) 

for j = 0, 1, . . . , J, A: = 0, . . . , 2-^ — 1, and J(< logj n) is the maximum i level of re- 
cursions specified by the user. Here, we have rij = dim = n/2-^. The wavelet 
packet transforms, recursively split the frequency domain via the so-called con- 
jugate quadrature filters. These splits naturally generate a set of subspaces with 
the binary tree structure. Let a = (j,k) be an index to specify a node (i.e., a 
subspace with its basis set) of this binary tree. The index j specifies the depth 
of the binary tree; this is an index of the width of frequency bands for wavelet 
packets. The index k specifies the location of the frequency bands for wavelet 
packets. Let be such a collection of subspaces and let B = {Ba} be the 

corresponding set of basis vectors where Ba = {tOai , • • • , ^auj ) is a set of basis 
vectors that spans fia- Each basis vector in B is called a time-frequency atom, 
and the whole set B is referred to as a time-frequency dictionary or a dictionary 
of orthonormal bases. These dictionaries contain many orthonormal bases. If the 
depth of the tree is J, each dictionary contains more than 2^*'^ '' different bases. 

An important question is how to select from a large number of bases in the 
dictionary a basis which performs ’’best” for one’s task. In order to measure the 
performance of each basis, we need a measure of efficiency or fitness of a basis for 
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the task at hand. For this purpose, several so-called information cost functionals 
have been proposed. A commonly used information cost functional is entropy. 
The entropy of a vector d = {d{k)} is defined by, 



n{d) 
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r 1092 I 




(2) 



This cost functional was used as the objective function to drive the evolutionary 
optimization approach outlined in Sect. 3. The goal of the optimization approach 
is to find an optimal, or quasi-optimal (in some sense) basis representation for a 
given dataset. 

The above considerations are the fundamentals behind the SBB algorithm 
[6]. However, it is possible to extend the library of bases in which the best 
representations are searched for, by introducing additional degrees of freedom 
that adjust the time-localization of the basis functions [2]. One such extension 
is the shift-invariant wavelet packet transform. Actually, one well-known disad- 
vantage of the discrete wavelet and wavelet packet transforms is the lack of shift 
invariance. The added dimension in the case of shift-invariant decompositions 
is a relative shift, between a given parent-node and its relative children nodes. 
Shifted versions of these transforms for a given input signal, represent new bases 
to be added to the library of bases, which may further improve our ability to 
find the ’’best” adapted basis. These modifications of the wavelet transform and 
wavelet packet decompositions lead to orthonormal best-basis representations 
which are shift-invariant and characterized by lower information cost function- 
als [5]. Wavelet packet trees may be extended as joint wavelet packet trees to 
profit from enlarged libraries of bases, thus increasing our chances of getting truly 
adapted waveform representations. However, enlarged search spaces entail com- 
binatorial explosion problems. We rely on evolutionary optimization approaches 
to guide us on the search process. 



3 Evolutionary Formulation 

The two major major steps in applying any heuristic search algorithm to a par- 
ticular problem are the specification of the representation and the evaluation 
(fitness) function. When defining an evolutionary algorithm one needs to choose 
its components, such as mutation and recombination that suit the representa- 
tion, selection mechanisms for selecting parents and survivors, and an initial 
population. Each of these components have parameter values which determine 
whether the algorithm will find an near-optimum solution and whether it will 
find a solution efficiently. 

3.1 Representation 

In the work reported here, a variable length integer sequence is used as the basic 
genotype. The objective is to evolve wavelet decomposition trees through the 
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evolution of genetic sequences. The technique used to initialize the population is 
based on generating an initial random integer sequence, according to the values 
of the allele sets specified for the individual genes. The initial genotype sequence 
which codifies the wavelet tree matches the breadth-first (BF) sequence required 
to generate a complete binary tree, up to a pre-specified maximum depth. We 
refer to these sequences as tree-mapped sequences. A well-built decomposition 
tree for wavelet analysis purposes, is generated by imposing appropriate con- 
straints to the genotype sequence as specified in Sect. 3.2. The imposition of 
the constraints yields variable length code sequences after resizing. An alphabet 
A = {0,1,2} is used to codify the wavelet tree nodes according to their types 
as specified in Sect. 3.3, thus enabling us to map any tree structure into a code 
sequence. The mapping of a code sequence to a complete BF tree traversal yields 
an initial sequence with length I = 2^ — 1, for a tree of depth L. The length I 
is also the number of nodes in a complete binary tree of depth L. When coding 
a complete binary tree using a complete BF sequence the last level of termi- 
nal nodes is redundant. Therefore, we have used a codification based on resized 
complete BF sequences to code genetic sequences. The chromossomes are con- 
structed as follows. The first gene assumes integer values go € F, where F is the 
set of possible filter types used in the implementation, as explained in Sect. 3.3. 
The remaining genes are used to codify the wavelet decomposition tree. 



3.2 Constraints 

There are several methods for generating trees which can be used to initialize 
the population. The full, grow and ramped half-and-half methods of tree genera- 
tion were introduced in the field of genetic programming [9] . These methods are 
based on tree depth. The ramped half-and-half method is the most commonly 
used method of generating random parse trees because of its relative higher prob- 
ability of generating subtrees of varying depth and size. However, these methods 
do not produce a uniform sampling of the search space. In this work, we use 
constrained genetic sequences for genome initialization. Two types of constraint 
operators are used to guarantee that valid tree-mapped genetic sequences are 
generated: (1) top-down operator, and (2) bottom-up operator. In addition, by 
applying these operators we look for a uniform sampling of the tree search space. 
In terms of binary tree data structures, the top-down constraint guarantees that 
if a node A has null code c, = 0 then its two sons and Ap must have null code 
Cifi = 0 and c,p =0. The bottom-up constraint guarantees that if at least one 
of the sons and Ap of a node A has non-null code, then the parent must 
have non-null code c, 0. These constraint operators are biased in opposite 
ways. Starting from a uniform random code sequence, the bottom-up constraint 
operator constructs valid genetic sequences which are biased towards complete, 
full-depth trees. By he same token, the top-down constraint operator constructs 
valid genetic sequence which is biased towards null, minimum depth trees. To 
get a more uniform sampling of the sequence space, for sequences of maximum 
s = 2^ — 1, we use the following initialization procedure: 
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InteriorNodes Depth 



Fig. 1. Fiistogram for the distribution of interior nodes in generated sequences (left 
panel); histogram for the distribution of trees with specified depth (right panel) 



1. initialize sequence s to terminal code value c, = 0 € A, 

2. get a random value r € [l,s], 

3. initialize subsequence si = [l,r) € s with random interior code values, 
Ci = {1,2} G A, 

4. randomly select one of the two constraint operators, bottom-up or top-down, 
to apply to s. 

By resizing (pruning) constrained code sequences we allow for genetic se- 
quences of variable length, hence tree representations of variable depth. The left 
panel in Fig. 1 shows a histogram for the number of interior nodes generated 
by the initialization procedure, for a maximum specified depth L = 8 of the 
equivalent tree, and 3000 stochastic genetic sequences. The right panel in Fig. 1 
presents a histogram for the distribution of trees with specified depth, generated 
from the same stochastic samples with resizing. 



3.3 Specification 

The approach of organizing libraries of bases as a tree has been extended to 
construct a joint tree, to guide the process of generating shifted wavelet packet 
transforms. Libraries of bases represent the population from which we want to 
select the best-fit individuals. In the current formulation, the genotype sequence 
Q allows for three optimization parameters: best filter, best wavelet packet basis 
and best shifted basis. The genetic representation is used to create an initial pop- 
ulation and evolve potential solutions to the optimization problem. The genotype 
is made up of the genes which guide the discrete wavelet decomposition of each 
waveform, in accordance with the joint tree representation. A cost functional 
is then applied to the wavelet coefficients, and its value is used to derive the 
fitness of the individual. In terms of entropy, the optimization problem amounts 
to evolve a minimum-entropy genotype. Therefore, the best individual is the 
one with minimum evolved entropy in a given library space. Since we were able 
to formulate the three subtasks, wavelet packet decomposition, shifted wavelet 
transform, and wavelet filter to be applied in a common data structure, the orig- 
inal multiple optimization problem can be solved in terms of a single aggregate 
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functional. The first gene go in Q is responsible for the optimization of the filter 
used in the decomposition. We have used in the implementation 16 possible types 
of different filters, thus gio = {0, . . . , 15}. The filters considered in the implemen- 
tation were the Haar filter, the Daubechies filters D4, D6 and D8, and several 
biorthogonal filters commonly used in image analysis as implemented in [7]. In 
particular, the filter set included the 7/9 spline filter referred to in [1], the 7/9, 
11/13, 10/6, 3/5, 6/2, and 3/9 filters defined in [11], the 7/9 ” FBI-fingerprint” 
filter, and the 10-tap filter listed in [3]. 

The analysis phase of the (discrete) shift wavelet packet transform is codified 
in the genetic sequence Q. The collection of wavelet packets comprises a library 
of functions with a binary tree structure. To obtain the wavelet packet analysis of 
a function, or data set in the discrete case, we first find its coefficient sequence in 
the root subspace, then follow the branches of the wavelet packet coefficient tree 
to find the expansion in the descendent subspaces. Assigning to each tree node 
a wavelet split value s, € {0,1,2} we may enumerate all possible binary tree 
structures. The value s™ = 1 references unshifted interior nodes, i.e., nodes with 
left and right children subtrees associated with unshifted decompositions. The 
value Sw =2 references time shifted interior nodes. The value = 0 references 
the leaves. 



4 Spaces of Test Signals 

A well-known implementation of the SBB algorithm which can be used for com- 
parison purposes is contained in the WaveLab package [4] . We reference by evo- 
lutionary best basis (EBB), the evolutionary formulation presented in Sect. 3 
for best basis selection using multifilter, time shifted wavelet packet bases. By 
eanonieal best basis (CBB), we mean an algorithm which is able to reproduce 
(approximate) the results of the SBB algorithm, using optimization methodolo- 
gies for best basis selection different from those conceived for the SBB algorithm. 
In this sense, we may map the EBB algorithm into a CBB algorithm. The SBB 
algorithm is based on building an information cost tree in order to minimize 
some cost measure on the transformed coefficients [6]. The evolutionary algo- 
rithm proposed in Sect. 3 was applied to a set of test signals and the results 
compared with the results produced by SBB, based on the entropy minimiza- 
tion criterion. Eor evaluation purposes, we will use the test signals depicted in 
figure 2. Two of these signals are artificial signals. The other two signals are 
built from collected data. The signal HypChirps includes two hyperbolic chirps. 
The signal MishMash includes a quadratic chirp, a linear chirp, and a sine, as 
used in the WaveLab package. The signal Seismie is distributed throughout 
the seismic industry as a test dataset. Einally, the signal Sunspots represent the 
monthly sunspot numbers. The following basic parameters have been used in the 
steady-state evolutionary algorithm [8,12]: (1) population size: 50; (2) crossover 
probability: 0.95; (3) mutation probability: 0.02; (4) replacement percentage: 0.6. 
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Fig. 2. Test signals 



4.1 Standard Wavelet Spaces 

To reproduce the application conditions of the SBB algorithm, the EBB ap- 
proach was restricted to handle unshifted wavelet packet decompositions and 
use a specific filter. Hence, the CBB algorithm is a restricted version of the EBB 
algorithm, which is used to reproduce the SBB results using a different method- 
ology. Both the EBB and the CBB algorithms are evolutionary. The minimum 
entropy values associated with the best basis selected by the SBB (WaveLab) 
algorithm for these signals, using the Daubechies D8 filter and L = 9 decompo- 
sition levels, are reported in Table 1. The SBB entropy values are also depicted 
by the dashed lines in Eig. 3. The CBB algorithm was applied to the same set 
of test signals to evolve the best basis using the same entropy cost functional. 
Eig. 3 shows the evolution of the median minimum entropy (median best value) 
with the number of generations for each of the test signals. The entropy values 
represent the median over 30 runs of the CBB algorithm. The median entropy 
values after ngen = 80 generations are shown in Table 1 as well. The optimal 
entropy values and the rate of convergence can be used as benchmarks to com- 
pare the performance of different evolutionary algorithms, or simply to tune the 
value of the control parameters. 



Table 1. Comparative median minimum entropy values and reconstruction errors 



Signal Median Minimum Entropy (L=9) Reconst. Error 



SBB CBB EBB EBB 

(D8) (D8, ngen=80) (ngen=80) (ngen=80) 



HypChirps 3.7908 


3.7908 


2.7707 


1.0 e-6 


MishMash 


4.4805 


4.5162 


3.5910 


2.6 e-6 


Seismic 


2.6022 


2.6025 


1.9735 


2.7 e-7 


Sunspots 


3.0059 


3.0059 


2.7749 


3.9 e-8 
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Fig. 5. Evolved trees by the EBB algorithm 



4.2 Enlarged Wavelet Spaces 

To show the application of the EBB technique we have applied it to evolve basis 
for the optimization of three parameters: wavelet packet decomposition, shiftable 
decomposition and filter to use. Fig. 4 presents the evolution of the minimum 
entropy values generated by the EBB algorithm for the signals in the test set. The 
values are the median values over 30 runs of the EBB algorithm. The dashed 
horizontal line in Fig. 4 is the value of the best basis entropy generated by 
SBB. We notice that the EBB algorithm is able to greatly reduce the minimum 
entropy value used to assess best basis adaptability, compared to both the SBB 
and the CBB algorithms. Table 1 references the numeric median values for the 
EBB minimum entropy after ngen = 80 generations. The most selected filters 
among the best evolved filters for each test signal, were the following: Brislawn 
10-tap filter for signals HypChirps and MishMash, Villasenor 3/9 filter for signal 
Seismic, and Villasenor 6/2 filter for signal Sunspots. Fig. 5 depicts typical best 
evolved trees for a sample run of the EBB algorithm. Darker lines represent 
shifted wavelet packet transforms. Thinner lines represent unshifted transforms. 

Another important evaluation factor is the reconstruction error. Given a sig- 
nal /, we reconstruct an approximate signal / from the transformed coefficients 
by applying the inverse shifted wavelet packet transform, and calculating the I 2 
error between these two signals || f — f H 2 . Table 1, presents the numeric median 




Evolutionary Wavelet Bases in Signal Spaces 53 



values of the reconstruction errors for each of the test signals, using the EBB 
algorithm for 80 generations over 30 runs. 



5 Conclusion 

The approximation of signals in functions spaces was used to introduce a test 
environment aimed at the comparative performance of different evolutionary al- 
gorithms. We have considered entropy as the optimization fitness criterion to be 
used. However, other cost functions may prove useful to extend the range of the 
test environment. In particular, measures directed to the optimization of multi- 
objective criteria may be incorporated in the proposed framework. On the other 
hand, the proposed test environment may be easily extended to incorporate two 
dimensional signal spaces. In terms of signal processing, well-adapted signal ex- 
pansions are important, for instance, in signal compression. For orthonormal 
basis and additive cost measures, the standard algorithm for best basis selection 
is efficient. However, with the introduction of overcomplete waveform dictionaries 
the algorithm has increasing difficulty in finding well-adapted signal represen- 
tations. The proposed evolutionary approach offers more flexibility in searching 
for well-adapted signal representations than standard approaches. 



References 

1. M. Antonini, M. Barlaud, P. Mathieu, and I. Daubechies. Image coding using 
wavelet transform. IEEE Trans, on Image Process., 1(2), April 1992. 

2. G. Beylkin. On the representation of operators in bases of compactly supported 
wavelets. Society for Industrial and Applied Mathematics, 6(6):1716-1740, Decem- 
ber 1992. 

3. C. M. Brislawn. Two-dimensional symmetric wavelet transform tutorial program. 
Technical report, Los Alamos National Laboratory, December 1992. 

4. J. Buckheit and D. L. Donoho. Wavelab and reproducible research. Technical 
report. Department of Statistics, Stanford University, 1995. 

5. I. Cohen, S. Raz, and D. Malah. Orthonormal shift-invariant wavelet packet de- 
composition and representation. Signal Processing, 57(3):251-270, March 1997. 

6. R. R. Coifman and M. V. Wickerhauser. Entropy based methods for best basis 
selection. IEEE Trans, on Inf. Theory, 38(2):719-746, 1992. 

7. G. Davis. Baseline Wavelet Transform Coder Constrution Kit. Mathematics De- 
partment, Dartmouth College, January 1997. 

8. D. E. Goldberg. Genetic Algorithms in Search, Optimization, and machine learning 
and Filter Banks. Addison- Wesley, Reading, Massachusetts, 1989. 

9. John R. Koza. Genetic Programming - On the Programming of Computers by 
Means of Natural Selection. MIT Press, Cambridge, MA, 1992. 

10. Z. Michalewicz. Genetic algorithms + data structures = evolution programs. Arti- 
ficial Intelligence. Springer- Verlag, New York, 1992. 

11. J. Villasenor, B. Belzer, and J. Liao. Wavelet filter evaluation for image compres- 
sion. IEEE Trans, on Image Process., 4(8):1053-1060, August 1995. 

12. M. Wall. GAlib: A C++ Library of Genetic Algorithm Components. Mechanical 
Engineering Department, Massachusetts Institute of Technology, August 1996. 




Finding Golf Courses: The Ultra High Tech 

Approach 



Neal R. Harvey, Simon Perkins, Steven P. Brumby, James Theiler, 
Reid B. Porter, A. Cody Young, Anil K. Varghese, John J. Szymanski and 

Jeffrey J. Bloch 

Space and Remote Sensing Sciences Group, 

Los Alamos National Laboratory, Los Alamos, NM 87545, USA 



Abstract. The search for a suitable golf course is a very important issue 
in the travel plans of any modern manager. Modern management is also 
infamous for its penchant for high-tech gadgetry. Here we combine these 
two facets of modern management life. We aim to provide the cutting- 
edge manager with a method of finding golf courses from space! 

In this paper, we present Genie: a hybrid evolutionary algorithm-based 
system that tackles the general problem of finding features of interest in 
multi-spectral remotely-sensed images, including, but not limited to, golf 
courses. Using this system we are able to successfully locate golf courses 
in 10-channel satellite images of several desirable US locations. 



1 Introduction 

There exist huge volumes of remotely-sensed multi-spectral data from an ever- 
increasing nnmber of earth-observing satellites. Exploitation of this data requires 
the extraction of features of interest. In performing this task, there is a need for 
snitable analysis tools. Creating and developing individual algorithms for specific 
feature-detection tasks is important, yet extremely expensive, often requiring a 
significant investment of time by highly skilled analysts. To this end we have 
been developing a system for the automatic generation of useful feature-detection 
algorithms using an evolutionary approach. 

The beauty of an evolutionary approach is its flexibility: if we can derive a 
fitness measure for a particular problem, then it might be possible to solve that 
problem. Many varied problems have been successfully solved using evolution- 
ary computation, including: optimization of dynamic routing in telecommunica- 
tions networks [1] , optimizing image processing filter parameters for archive film 
restoration [2] , designing protein sequences with desired structures [3] and many 
others. 

When taking an evolutionary approach, a critical issue is how one should 
represent candidate solutions in order that they may be effectively manipulated. 
We use a genetic programming (GP) method of representation of solutions, due 
to the fact that each individual will represent a possible image processing algo- 
rithm. GP has previously been applied to image-processing problems, including: 
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edge detection [4] , face recognition [5] , image segmentation [6] , image compres- 
sion [7] and feature extraction in remote sensing images [8-10]. The work of 
Daida et al. Brumby et al. and Theiler et al. is of particular relevance since it 
demonstrates that GP can be employed to successfully evolve algorithms for real 
tasks in remote-sensing applications. 

2 System Overview 

We call our feature detection system “Genie” (GENetic Image Exploitation) [9, 
10] Genie employs a classic evolutionary paradigm: a population of individuals 
is maintained and each individual is assessed and assigned a fitness value. The 
fitness of an individual is based on an objective measure of its performance 
in its environment. After fitness determination, the evolutionary operators of 
selection, crossover and mutation are applied to the population and the entire 
process of fitness evaluation, selection, crossover and mutation is iterated until 
some stopping condition is satisfied. 



2.1 Training Data 

The environment for each individual in the population consists of a set of train- 
ing data. This training data consists of a data “cube” of multi-spectral data 
together with some user-defined data defining “ground-truth” . Ground-truth, 
in this context, is not what is traditionally referred to as ground-truth (this 
being in-situ data collected at, or as close as possible to, the time the image 
was taken). Here, ground-truth refers to what might normally be referred to as 
“analyst-supplied interpretation” or “training data” . This training data for our 
system is provided by a human analyst, using a Java-based tool called Aladdin. 
Through Aladdin, the user can view a multi-spectral image in a variety of ways, 
and can “mark up” training data by “painting’ directly on the image using the 
mouse. Training data is ternary- valued with the possible values being “true” , 
“false” , and “unknown” . True defines areas where the analyst is confident that 
the feature of interest does exist. False defines areas where the analyst is confi- 
dent that the feature of interest does not exist. Fig. 1 shows a screen capture of 
an example session. Here the analyst has marked out golf courses as of interest. 



2.2 Encoding Individuals 

Each individual chromosome in the population consists of a fixed-length string 
of genes. Each gene in Genie corresponds to a primitive image processing op- 
eration, and so the whole chromosome describes an algorithm consisting of a 
sequence of primitive image processing steps. 



Genes and Chromosomes A single gene consists of an operator name, plus 
a variable number of input arguments, specifying where input is to come from; 
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Fig. 1. GUI for Training Data Mark-Up. Note that Aladdin relies heavily on color, 
which does not show up well in this image. The light colored patches in the center-right 
and upper-right parts of the image are two golf courses that have been marked up as 
“true” . Most of the rest of the image has been marked up as “false” , except for a small 
region around the golf courses which has been left as “unknown” . 



output arguments, specifying where output is to be written to; and operator 
parameters, modifying how the operator works. Different operators require dif- 
ferent numbers of parameters. The operators used in Genie take one or more 
distinct image planes as input, and generally produce a single image plane as 
output. Input can be taken from any data planes in the training data image 
cube. Output is written to one of a small number of scratch planes — temporary 
workspaces where an image plane can be stored. Genes can also take input from 
scratch planes, but only if that scratch plane has been written to by another 
gene positioned earlier in the chromosome sequence. 

The image processing algorithm that a given chromosome represents can 
be thought of as a directed acyclic graph where the non-terminal nodes are 
primitive image processing operations, and the terminal nodes are individual 
image planes extracted from the multi-spectral image used as input. The scratch 
planes are the ‘glue’ that combines together primitive operations into image 
processing pipelines. Traditional GP ([11]) uses a variable sized (within limits) 
tree representation for algorithms. Our representation differs in that it allows 
for reuse of values computed by sub-trees since many nodes can access the same 
scratch plane, i.e. the resulting algorithm is a graph rather than a tree. It also 
differs in that the total number of nodes is fixed (although not all of these may 
be actually used in the final graph) , and crossover is carried out directly on the 
linear representation. 

We have restricted our “gene pool” to a set of useful primitive image process- 
ing operators. These include spectral, spatial, logical and thresholding operators. 
Table 1 outlines these operators. For details regarding Laws textural operators, 
the interested reader is referred to [12, 13]. 

The set of morphological operators is restricted to function-set processing 
morphological operators, i.e. gray-scale morphological operators having a flat 
structuring element. The sizes and shapes of the structuring elements used by 
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Table 1. Image Processing Operators in the Gene Pool 



Code 


Operator Description 




Code 


Operator Description 


ADDP 


Add Planes 




MEAN 


Local Mean 


SUBP 


Subtract Planes 




VARIANCE 


Local Variance 


ADDS 


Add Scalar 




SKEWNESS 


Local Skewness 


SUBS 


Subtract Scalar 




KURTOSIS 


Local Kurtosis 


MULTP 


Multiply Planes 




MEDIAN 


Local Median 


DIVP 


Divide Planes 




SD 


Local Standard Deviation 


MULTS 


Multiply by Scalar 




EROD 


Erosion 


DIVS 


Divide by Scalar 




DIL 


Dilation 


SQR 


Square 




OPEN 


Opening 


SQRT 


Square Root 




CLOS 


Closing 


LINSCL 


Linear Scale 




OPCL 


Open-Closing 


LINCOMB 


Linear Combination 




CLOP 


Close-Opening 


SOBEL 


Sobel Gradient 




OPREC 


Open with Reconstruction 


PREWITT 


Prewitt Gradient 




CLREC 


Close with Reconstruction 


AND 


And Planes 




HDOME 


H-Dome 


OR 


Or Planes 




HBASIN 


H-Basin 


CL 


Clip Low 




CH 


Clip High 


LAWB 


Laws Textural Operator 53^ 


X L3 


LAWC 


Laws Textural Operator L3^ x E3 


LAWD 


Laws Textural Operator 


X EZ 


LAWE 


Laws Textural Operator S2»^ x EZ 


LAWF 

LAWH 


Laws Textural Operator L3^ 
Laws Textural Operator So 


X S3 
X S3 


LAWC 


Laws Textural Operator E3^ x S3 



these operators is also restricted to a pre-defined set of primitive shapes, which 
includes, square, circle, diamond, horizontal cross and diagonal cross, and hori- 
zontal, diagonal and vertical lines. The shape and size of the structuring element 
are defined by operator parameters. Other local neighborhood/windowing oper- 
ators such as mean, median, etc. specify their kernels/windows in a similar way. 
The spectral operators have been chosen to permit weighted sums, differences 
and ratios of data and/or scratch planes. 

We use a notation for genes that is most easily illustrated by an example: the 
gene [ADDP rDO rSl wS2] applies pixel- by-pixel addition to two input planes, 
read from data plane 0 and from scratch plane 1, and writes its output to scratch 
plane 2. Any additional required operator parameters are listed after the input 
and output arguments. 

Note that although all chromosomes have the same fixed number of genes, 
the effective size of the resulting algorithm graph may be smaller than this. 
For instance, an operator may write to a scratch plane that is then overwritten 
by another gene before anything reads from it. Genie performs an analysis of 
chromosome graphs when they are created and only carries out those processing 
steps that actually affect the final result. Therefore, in some respects, we could 
refer to the fixed length of the chromosome as a “maximum” length. 



2.3 Backends 

Complete classification requires that we end up with a single binary-valued out- 
put plane from the algorithm. It would be possible to treat, say, the contents of 
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scratch plane 0 after running the chromosome algorithm, as the final output from 
the algorithm (thresholding would be required to obtain a binary result). How- 
ever, we have found it to be of great advantage to perform the final classification 
using a non-evolutionary algorithm. 

To do this, we first select a subset of the scratch planes and data planes 
to be answer planes. Typically in our experiments this subset consists of just 
the scratch planes. We then use the provided training data and the contents of 
the answer planes to derive the Fisher Discriminant, which is the linear com- 
bination of the answer planes that maximizes the mean separation in spectral 
terms between those pixels marked up as “true” and those pixels marked up 
as “false”, normalized by the “total variance” in the projection defined by the 
linear combination. See [14] for details of how this discriminant works. 

The output of the discriminant-finding phase is a gray-scale image. This is 
then reduced to a binary image by using Brent’s method [15] to find the threshold 
value that minimizes the total number of misclassifications (false positives plus 
false negatives) on the training data. 



2.4 Fitness Evaluation 

The fitness of a candidate solution is given by the degree of agreement between 
the final binary output plane and the training data. This degree of agreement 
is determined by the Hamming distance between the final binary output of the 
algorithm and the training data, with only pixels marked as true or false con- 
tributing towards the metric. The Hamming distance is then normalized so that 
a perfect score is 1000. To put this in a more formal/mathematical context. Let 
H be the Hamming distance between the final binary output of the algorithm 
and the training data, with only pixels marked as true or false contributing to- 
wards the metric, let N be the number of classified pixels in the training image 
(i.e. pixels marked as either “true” or “false”) and let F be the fitness of the 
candidate solution. 



F = (1 - (H/N)) X 1000 (1) 

2.5 Software Implementation 

The genetic algorithm code has been implemented in object-oriented Perl. This 
provides a convenient environment for the string manipulations required by the 
evolutionary operations and simple access to the underlying operating system 
(Linux). Chromosome fitness evaluation is the computationally intensive part of 
the evolutionary process and for that reason we currently use RSI’s IDL language 
and image processing environment. Within IDL, individual genes correspond to 
single primitive image operators, which are coded as IDL procedures, with a chro- 
mosome representation being coded as an IDL batch executable. In the present 
implementation, an IDL session is opened at the start of a run and communi- 
cates with the Perl code via a two-way unix pipe. This pipe is a low-bandwidth 
connection. It is only the IDL session that needs to access the input and training 
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data (possibly hundreds of Megabytes), which requires a high-bandwidth con- 
nection. The Aladdin training data mark-up tool was written in Java. Fig. 2 
shows the software architecture of the system. 




Fig. 2. Software Architecture of the System Described. Note that the feature depicted 
on the right of this diagram represents the input data, training data and scratch planes 



3 Why Golf Courses? 

The usefulness of devising algorithms for the detection of golf courses may not, 
at first, seem apparent (except to a manager, perhaps!). However, due to the 
nature of golf courses and their characteristics in remotely-sensed data, they are 
of great use in testing automatic feature-detection systems, such as described 
here. They possess distinctive spectral and spatial characteristics and it is the 
ability of feature-detection algorithms to utilize both these “domains” that we 
seek to test. It is also useful that there exists a great deal of “ground truth” 
data available: a great many golf courses, for the benefit of low-tech managers, 
are marked on maps. In addition, golf courses usually possess a well-known, 
particular type of vegetation and it is rare to find information regarding specific 
vegetation types on maps. Fig. 3 (a) shows a map of NASA’s Moffet Field Air 
Base, clearly showing the position of a golf course. Fig. 3 (b) shows a false col- 




(a) (b) 



Fig. 3. (a) Map of NASA’s Moffet Field Air Base, showing a golf course (available at 
http:/ /george. arc. nasa.gov/jf/mfa/thesite2.html) (b) Image from remotely-sensed data 
of NASA’s Moffet Field Air Base 
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our image of some remotely sensed data of the same region. The airfield and golf 
course are clearly visible. 

4 Remotely-Sensed Data 

The remotely-sensed images referred to in this paper are 10-channel simulated 
MTI data, produced from 224-channel AVIRIS data, each channel having 614 x 
512 pixels. The images displayed are false-color images (which have then been 
converted to gray-scale in the printing process). The color mappings used are 
the same for all images shown (an exception being Fig. 1 where the false-color 
image has had a red and green overlay, corresponding to “false” and “true” 
pixels, as marked by the human analyst). The particular color mappings used 
here involve averaging bands A and B for the blue component, bands C and D 
for the green component and bands E and F for the red component. In addition, 
the images have been contrast enhanced. The choice of color mappings was 
arbitrary, in that it was a personal decision made by the analyst, made in order 
to best “highlight” the feature of interest, from his/her perspective and thus 
enable him/her to provide the best possible training data. This choice of color- 
mappings, together with a contrast-enhancement tool, are important and very 
useful features of Aladdin. Table 2 provides details about MTI data. 



Table 2. MTI Band Characteristics 



Band 


Wavelength (/rm) 


Color 


SNR 


Ground Sample Distance 


A 


0.45-0.52 


blue/green 


120 


5m 


B 


0.52-0.60 


green/yellow 


120 


5m 


C 


0.62-0.68 


red 


120 


5m 


D 


0.76-0.86 


NIR 


120 


5m 


E 


0.86-0.89 


NIR 


500 


20m 


F 


0.91-0.97 


NIR 


300 


20m 


G 


0.99-1.04 


SWIR 


600 


20m 


H 


1.36-1.39 


SWIR 


4 


20m 


I 


1.55-1.75 


SWIR 


700 


20m 


O 


2.08-2.35 


SWIR 


600 


20m 


J 


3.50-4.10 


MWIR 


250 


20m 


K 


4.87-5.07 


MWIR 


500 


20m 


L 


8.00-8.40 


LWIR 


800 


20m 


M 


8.40-8.85 


LWIR 


1000 


20m 


N 


10.2-10.7 


LWIR 


1200 


20m 



Figs. 3(a), 4(a) and 5(a) are data taken over an area of NASA’s Moffet Field 
Air Base in California, USA. Fig. 3(a) is a sub-set of the data shown in Fig. 4(a). 
Figs. 3(a) and 5(a) are non-adjacent regions of the original data. These sub-sets 
of the data contain a lot of different features, but, of course, have a common 
feature of interest: golf courses. 
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5 Searching for Golf Courses 



We reserve the data described above (Fig. 3(a)) for testing an evolved golf- 
course finder algorithm and set the system the task of finding a golf course on 
some other data. This data, showing the “truth” as marked out by an analyst, 
is shown in Fig. 1. The golf course area has been marked as “true” and most 
of the remaining data has been marked as “false” . The system was run for 400 
generations, with a population of 100 chromosomes, each having a fixed length of 
20 genes. At the end of the run the best individual had a fitness of 966 (a perfect 
score would be 1000). This fitness score actually translates into a detection rate 
of 0.9326 and a false alarm rate of 0.00018. The results of applying the best 
overall algorithm found during the run to the data used in the training run are 
shown in Fig. 4. 




Fig. 4. (a) Image of training data (b) Result of applying algorithm found to training 
data 



It can be seen that the algorithm has been able to successfully detect the 
golf course and has not detected any of the other features within the image. 

In order to test the robustness of the algorithm found, it was applied to out- 
of-training-sample data, as described previously, and shown in Fig. 3 (b). The 
results are shown in Fig. 5. 

It should be noted that the data shown in Fig. 5 covers a greater area than 
shown by the map in Fig. 3 (a). It can be seen that the algorithm has successfully 
found the golf course shown on the map. It can also be seen that the algorithm 
has detected other golf courses. On closer examination of the data, it would 
appear that further golf courses do, in fact, exist at those locations. It can also 
be seen that the algorithm has not found any spurious features. 

The “short” (redundant genes stripped out) version of the chromosome found 
is detailed below. 

[LAWG rD2 wSO] [OPREC rD3 wS3 5 1] [ADDP rSO rS3 wSl] [ADDP rSl 
rD6 wSI] [LAWE rD6 wS4] [LAWG rD6 wSO] [OPCL rS4 wS3 I I] [DIL rSl wSl 
1 0] [OPREC rSl wSI 5 0] [MEDIAN rSI wS2 I] [LAWH rD2 wS4] 
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(a) (b) 



Fig. 5. (a) Image of out-of-training-sample data (b) Result of applying algorithm found 
to out-of-training-sample data 

A graphical representation of the algorithm found is shown in Fig. 6. Note 
that the circles at the top of the graph indicate the data planes input to the 
algorithm (in this case only 3 data planes out of a possible 10 have been selected), 
the 5 circles in the center represent the scratch planes and the circle at the bottom 
represents the final, binary output of the overall algorithm. The operations above 
the line of scratch planes represent that part of the overall algorithm incorporated 
in the chromosome. The operations below the line of scratch planes represent the 
optimal linear combination of scratch planes and intelligent thresholding parts 
of the overall algorithm. 

It is interesting to have some kind of objective measure of the algorithm’s 
performance on the out-of-training-sample data. To this end an analyst marked 
up training data (i.e. true and false) for this data, with respect to the golf courses 
present. This enabled determination of a fitness for the algorithm on this data as 
well as detection and false alarm rates. The fitness of the algorithm was 926.6, 
the detection rate was 0.8532 and false-alarm rate was 3.000E-05. 

6 Comparison with Other Techniques 

In order to compare the feature-extraction technique described here to a more 
conventional technique, we used the Fisher discriminant, combined with the in- 
telligent thresholding, as described previously, to try and extract the golf courses 
in the images shown/described. This approach is based purely on spectral in- 
formation. On application to the data used in the training run (Fig. 4(a)), this 
“traditional” approach produced a result having a fitness of 757.228 (with re- 
spect to the training data/analyst-supplied interpretation), which translates into 
a detection rate of 0.5159 and a false-alarm rate of 0.00141. On application to the 
out-of-training-sample data, the result had a fitness of 872.323, which translates 
into a detection rate of 0.7477 and false-alarm rate of 0.00305. Both of these 
results are significantly below the performance of the results produced by the 
Genie system described here. 
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7 Conclusions 

A system for the automatic generation of remote-sensing feature detection algo- 
rithms has been described. This system differs from previously described systems 
in that it combines a hybrid system of evolutionary techniques and more tradi- 
tional optimization methods. It’s effectiveness in searching for useful algorithms 
has been shown, together with the robustness of the algorithms discovered. It has 
also been shown to significantly out-perform more traditional, purely-spectral 
approaches. 







Fig. 6. Graphical representation of algorithm found 
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is 3 H.ostH a larg ra g of fr q ci s st b pr s tad 
a ig r sol tio it sa pi al is d d. 
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aki g t s facts i to acco t, a 6 kH sa pli g fr q c as d cid d 
po , bas do a CO pro is b t r d ci g t discr ti atio rror, a d 
rdcigt a o tofipt data for t g tic progra . 

ral so ds r r cord d si g diff r t p ri tal s t ps. a 

si gi g oic as r cord d it a pt ad of t robot (a c bic al i 

bo , op it top a d back). o s ts of 6 H sa toot as r 

r cord d it diff r t pi a . i co a ds spok b a a oic r 
also r cord d. s r cordi gs ar r f r d to as r cordi gs , a I a d II 

ad o a ds i t r st of t pap r. sa pi s r r cord d si g 6 bits. 

It s t p for r cordi g t ad as pt . pi a r ad of 
plastic, ad r si ilar to a pi a . o a ditor ca als r pr s t. 

al a si gi g a to as r cord d fro 6 1 distrib t d dir ctio s i 

t ori o tal pla . or t r cordi gait ad as 11 d it i s lati g 

at rial. ditor ca als it a dia t r of 7 a d a d pt of r 

ad o t of plastic, o r t s all pi a . 6 H sa toot a as 

tra s itt d fro a lo dsp ak r .2 t rs fro t ad. o ds r r cord d 
fro 6 dir ctio s q all spac d i t ori o tal pla . 

I t r cordi go a ds t ad as 11 d it i s lati g at rial, 

a d t robot as q ipp d it ars ad of od li g cla a d od 1 d aft r 

a ars. a ditor ca als r appro i at 1 2 lo g. accid t, 

t ar ca als diff r d so at i idt . o ds r r cord d it t d 
ad plac d i diff r t positio s i t roo .1 ac r cordi gs r ad 

fro 6 1 distrib t d dir ctio sit ori o tal pla . so d so rc 

as plac d it r o , t o, t r or fo r t rs fro t d ad. 

r cord d so ds r a al oic gi i g t co a ds alk, sta d, for ard, 

back, stop, rig t, 1 ft a d grab. 

I t r cordi g a II t sa ad as i r cordi go a ds as s d 

( c pt t at t ar ca als r ad q all id ). a toot a s it a 
fr q c of 6 H r r cord d it t ad plac d at t r diff r t 
locatio s i t roo , ,2 a d 3 fro t so d so rc . t so ds 

fro 6 dir ctio s 1 distrib t d i t ori o tal pla r r cord d. I 
t ot r t o positio s so ds fro i dir ctio s as r cord d. 

2.2 ic pr ra i r s ca i a i 

progra s ol d s d a ra sa pi d so d sig al as i p t. progra 

as it rat d it f dback of t o tp t fro t pr io s it ratio a d t 

t pair of so d sa pi s as i p t. i di id al progra ca b i d 
as a f ctio os i p ts a d o tp ts ar arra s. i p t co sists of t o 
ori s, t pr io s r al o tp ts, t o sa pi s a d a co sta t. o tp t 

CO sists of t to ori s a d t o r al o tp ts. 

ori s r pr s t t o tp t fro t i di id al s d as i p t i t 

t it ratio . ’r al o tp ts’ ar t o tp ts s d to al at t a gl to 

t so d so rc . s ar s d as f dback it sa a as t ori s. 

sa pi s ar a pair of i p t sa pi s fro t 1 ft a d rig t so d c a 1. 
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I itiall t i p ts i for of ori s a d pr io s r al o tp ts ar s t 
to ro. ri g s bs q t it ratio s, ori s a d pr io s r al o tp ts, ar 
s d as f dback. rst ti a i di id al is r it is gi t rst pair of 
sa pi s i t t ss cas as i p t, a d a pair of sa pi s is t s ppli d 
i ac it ratio . 

ca d scrib o t i di id als ar s d i ps do-cod , s g r 

sa p ss as ) 

sa p ss as ) 

r ) 

) 

a sa p sa p s ) 

sa p ss as ) 

sa p ss as ) 

aaa s srrpr sra ps 

s of a i di id al to d t r i t dir ctio to a so d so rc is s o 
i ps do-cod . 



t o r al o tp ts ’o t ’ a d ’o t2’ ar it rpr tdast -ad - 
coordi at i a cart sia coordi at s st , a d t a gl is calc lat d fro 

t is i for atio . is r pr s tatio is s gg st d b t go tr of t prob- 

1 (a a g lar r pr s tatio as also tri d). 

b r of it ratio s d d ca b sti at d fro t p sics of t 

probl . o s t I t i di id al ds to s at 1 ast a b r of sa pi s 
corr spo di g to t ti d la . a i al ti d la is appro i at 1 t 

ti it tak s t so d to tra 1 t dista c b t t ars, appro i at 1 

.26 s for a 9 c robot ad. is corr spo ds to 4.2 sa pi s for a sa pli g 
fr q c of 6 H . o s t II t i di id al ds to s at 1 ast o 
a 1 gt . or a so d of 4 H t is corr spo ds to 4 sa pi s. 4 H is 

11 b lo t fr q ci s of ig st i t sit i t a sp aki g oic . i c 

ig r fr q ci s ar ost i t r sti g for t H o or sa pi s t a to 

al at t II ar lik 1 to b d d. 

2. s s 

li ar s st ol i g bi ar ac i cod as s d [3]. s st 
op rat s it a fo r i di id al to r a t s 1 ctio si g st ad stat . 
d s r s d, tog t r it a arit tic f ctio s t. 
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t ss cas CO sists of a b r of pairs of sa pi s. s r c os 
ra do 1 fro a lo g r arra of sa pi s fro a r cordi g corr spo di g to a 
sp ci c dir ctio to t so d so rc . 11 dir ctio s r gi q al ig t 

i t trai i g s t. t ss cas s r c os ac ti a i di id al as 

al at d, to i pro g rail atio . 

t st s t as c os ra do 1 fro t larg r s t of sa pi s i t sa 

a as t trai i g s t. al atio as do o t ss cas s c os t 

sa a as t o s s d for trai i g. alidatio s t as s d to c ck o 
11 1 g tic progra g rali s to a diff r t s t t a t trai i g s t. 

rror of a i di id al as calc lat d as diff r c , t s ort st a 
aro d t circl , b t t corr ct a gl a d t a gl calc lat d fro t 

0 tp t of t g tic progra . t ss as d d b t folio i g pr s- 

sio : 

* = 

i 

r is t b r of t ss cas s i t trai i g s t a d t id corr - 

spo ds to a sp ci c t ss cas . 

3 s ts 

s cc ss of t ol d progra ari d sig i ca tl it t p ri tal 
s t p. 

r s Its fro p ri ts I to ar s o i g r s 2 to 7. ig r s 2 to 
5 s o t t ss of t b st i di id al al at d o t trai i g, t st a d 

alidatio s ts. also sot a rag a d dia t ss for t pop latio 

as a ol , a d t progra 1 gt of t b st g tic progra a d t a rag 

1 gt of t g tic progra s i t pop latio . 

i diagra si gr6adgr7sot agl calc lat d aft r 
ac it ratio d ri g t c tio of t g tic progra i p ri ts I 
ad .1 g r 6 o grap is s o i ac diagra a d i g r 7 t r 

grap s corr spo di g to so ds co i g fro diff r t dista c s ar s o .1 

t i diagra s t d sir d a s r ra g s fro -9 it rst to +9 i 
t last it a spaci g of 22.5 . al atio s r do o a t st s t t at 

as id tical to t trai i g s t. 

abl s o s t para t rs t at r ari d b t p ri ts. 

folio i g para t rs r id tical i all p ri ts: 

( ) g tic progra s r trai d o so ds co i g fro t dir ctio s 
= , 22 5 , 45 , 67 5 a d 9 . 

(2) 11 sa pi s r a pli d i all p ri ts i s c a a t at 

t a i al sa pi a plit d as id tical i all t ss cas s. ft r t is a - 

pli catio a s CO d a pli catio as p rfor d i all p ri ts c pt for 

p ri t . last a pli catio as c os ra do 1 b t .ad 

for ac t ss cas , to a oid t at t g tic progra 1 ar d to r cog i ac 
r cord d so d 1 . 
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p- 

r. 


ata for: 
rai i 


st 


Val. 


rr. 
rai . 


rr. 

st 


rr. 

Val. 


tti s: 
loops 




I 


a I 
(9) 


a I 
(9) 


- 


2 


- 7 


- 


2 


4 


II 


a 

(27) 


a 

( 8) 


a I 

(9) 


4 


34 - 62 


3 - 5 


4 


5 


III 


a I 
(9) 


a I 
( 8) 


a 

(9) 


8 


6 - 


4 - 56 


4 


38 


IV 


a I 
(9) 


a I 
(9) 


a 

( 8) 




3 


42 - 63 


[36, 45] 


27 


V 


a II 
(27) 


a II 
(27) 


a I 
(9) 

a 

(45) 


23 


24 - 28 


45 - 53 
7 


[36, 45] 


32 



a . tabl is a s ar of para t r s tti gs a d r s Its for p ri- 
ts I to . ol t o to fo r s o t so d 1 s s d as trai i g, t st a d 

alidatio s ts, a d t b r of so d 1 s s d. I col to s t 

t ss of t b st i di id al calc lat d o t trai i g, t st a d alidatio s ts 

ar s o . folio i g col s o s t b r of it ratio s of t g tic 

progra s. I p ri t I a d t b r of it ratio s as c os ra - 

do 1 i t gi i t r al. b rs i t last col sot b r 

of g ratio s ac p ri t as r . 



( ) pop latio CO sist d of 4 i di id als di id d i to 4 d s. 

( ) 9 % crosso r probabilit as s d. probabilit for tatio s 

to occ r as t sa . probabilit for o ologo s crosso r as 4 % a d 
t r as o igratio b t d s. rat of crosso r b t d s 
as %. 

( ) o 1 CO sta t a ailabl to t g tic progra s as -4 3 6 2 . 

o i di id al as alio dtob logrta 2 bts (32 blocks). 

( 6 ) I p ri 1 1 a d II t t ss as al at d o t t ss cas s fro 
ac r cord d so d 1 . I t ot r p ri ts t at b r as 
t t ss as al at d o t t st s t a d trai i g s t t t t ss cas s 

r tak fro ac r cord dso dlit tstad alidatio s ts. 

o r al s cc ss r obtai d t trai i g s t r fro cordi g 

rror for so d locali atio o t trai igst as rsallrta 
s 2 . is r s It as obtai d all i di id als r al at d o actl 

t sa t ss cas s. rror of 42 as obtai d t t ss cas s 

rcagdbt ac al atio of a i di id al. is i dicat s t at t 
g ticall ol d progra o rtrai dad ori d t t ss cas si st ad 

of sol i g t r al probl of so d locali atio . 
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generations 




generations 



s Its fro p ri t I. PP r dia ra sost 1 ti tsoft 

st i di id al (solid) a d t a ra 1 t (das d). lo r dia ra sost 
fit ss of t st i di id al o t trai i s t (solid) a d t t st s t (dott d), t 
a ra fit ss (das d) a d t dia fit ss (das dot). 




generations 



s Its fro p ri t II. pp r dia rasost 1 ti tsof 

t st i di id al (solid) a d t a ra 1 t (dott d). lo r dia ra s o s 
t fit ss of t st i di id al o t trai i s t (solid), t t st s t (dott d) a d 
t alidatio s t (das d lo r). It also sost a ra fit ss (das d pp r) a d 
t dia fit ss (das dot). 
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s Its fro p ri t III. pp r dia rasost 1 ti t 

t st i di id al (solid) a d t a ra 1 t (dott d). lo r dia ra s 
t fit ss of t st i di id al o t trai i s t (solid), t t st s t (dott d) 
t alidatio s t (das d lo r). It also s o s t a ra fit ss (das d pp r) 
t dia fit ss (das dot). 





s Its fro p ri t IV. pp r dia ra s o s t 1 t i t 

t st i di id al (solid) a d t a ra 1 t (dott d). lo r dia ra s 
t fit ss of t st i di id al o t trai i s t (solid), t t st s t (dott d) 
t alidatio s t (das d lo r). It also s o s t a ra fit ss (das d pp r) 
t dia fit ss (das dot). 



s of 
o s 
a d 
a d 



s of 
o s 
a d 
a d 
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6 s Its for p ri t IV. ac of t dia ra s s o s t a 1 calc lat d 
fro t o tp ts of t st i di id al o o fit ss cas fro t t st s t. a 1 

is calc lat d aft r ac c tio of t i di id al i t loop c pt for t first, 

t st s t as id tical to t trai i s t. I ac of t i dia ra s t ori o tal 
li sot corr ct a s r. 






60 




60 



100 

50 , ^ 



0 







o' 

0 20 40 




40 

0 20 40 



60 



7 s Its for p ri t IV. ac of t dia ra s s o s t a 1 calc lat d 
fro t o tp ts of t st i di id al o t r fit ss cas s fro t t st s t. 
t r fit ss cas s corr spo d fro so ds fro t sa dir ctio t fro diff r t 
dista c s. dista c s r (solid), 2 (dott d) a d 3 (das dot). a 1 
is calc lat d aft r ac c tio of t i di id al i t loop c pt for t first. 
I ac dia ra , t t st s t r id tical to t trai i s t. I ac of t i 
dia ra s t ori o tal li s o s t corr ct a s r. 
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rst r ark t at ca tio s o Id b obs r d co pari g t r s Its 

fro t diff r t p ri ts, si c t diff r t trai i g s ts r r cord d 
it diff r t o t r ars. r s Its of [4] s o t at if t s ap of t a 

pi aiscagd og,or abilit to locali so ds is dra aticall r d c d. 

t s start it p ri t . fact t at t t ss s for t b st 
i di id al al at d o t trai i g a d t st s ts ar al ost q all s all (s 

tabl ), a s t at t g tic progra g rali s to all so ds i t trai i g 

s t. is is also t cas for t ot r p ri ts c pt for p ri t II, as 

ca b s i g r s 2 to 5. 

I tabl o ca s t at t ol d progra i p ri t do s ot 
g rali to a a sp aki g oic ( o a ds) trai do sa toot 

as. is is ot s rprisi g si c it is ard for t g tic progra to 

locali a sa toot a so d co i g fro diff r t dir ctio sad dista c s, 

s g r 7. is g r also s o s t at t g tic progra do s ot g rali 
t at 11 to so ds CO i g fro diff r t dista c s. diffic It is probabl 
d to c o s fro alls a d f r it r i t roo . 

rai i g o a a oic as also diffic It, s g r 3. b st ol d 

progra as a t ss of 4 . t ss ca b co par d to t at obtai d fro 

a progra t at al a s gi s t co sta t o tp t al = . t t ss 

is 5 , ic s o s t at so 1 ar i g as occ rr d. 

I gr6o cas tatt agl fro o i di id al ari s p riodicall 

al at d o o t ss cas , il co rgi g to t rig t a s r. 
p riod is t sa as t at of t sa toot a t i di id al is al t d o . 

is i dicat s t at t g tic progra s s t II i so a . is p riodic 

patt r as 1 ss cl ar t g tic progra s r ol d o so d co i g 

fro diff r t dista c s, s g r 7. r aso for t is a b t at t g tic 
progra s st 1 ar to cop it cos, ic ak s it or diffic It to s 
t II . 

r 

a so t at it is possibl to ol as all ac i cod progra t at 

ca i t rpr t a proc ss d str a of sa pi d st r o so d it o t i s rtio 

of do ai k o 1 dg or ot r str ct r . ol d progra ca locali di- 

r ctio of a so d it ar i g s cc ss. arti cial so d t i di id als a 
si ilar p rfor a c to as as r d as a g lar r sol tio , b t c r - 
s arc r ai s for t is p rfor a c to old dr or r alistic s tti gs a d 
CO ditio s. 

c ts 

is r s arc as s pport d b K, t 

lop t. 



dis oard for c ological 
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2. la rt . i ri h s h h si s i i , \ 

r ss, 997. 

3. a af ., t.al. i r r i , dp kt. rla ad or a a f a 

lis rs, H id r a d a ra sico, 998. 

4. Hof a . ., t.al., 1 ar i so d locali atio it ars, r r - 

si ,5, pp. 4 7-42 , 998. 
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. atur 1 cti n is a r pr ising pti isati n strat g 

f r att rn c gniti n s st s. ut, as an -c pi t task, it is 

tr 1 difficult t carr ut. ast studi s t r f r r rat r li it d 
in it r t cardinalit ft f atur spac r t nu r f patt rns 
utilis d t ass ss t f atur su s t p rf r anc . 

is stud a in s t scala ilit f istri ut d n tic Ig rit s 
t r larg -seal atur 1 cti n. s d ain f applicati n, a classi- 
ficati n s st f r ptical aract rs is c s n. s st is tail r d 
t classif and- ritt n digits, in 1 ing 76 inar f atur s. u t t 

astn ss f t in stigat d pr 1 , t is stud f r s a st p int n 

r al s in atur 1 cti n f r classificati n. 

pr s nt a s t f cust isati ns f s t at pr id f r an app- 
licati n f kn n c nc pts t atur 1 cti n pr 1 s f practical 
int r st. li itati ns f s in t d ain f atur 1 cti n ar 

unr al d and i pr nts ar sugg st d. id 1 us d strat g t 
acc 1 rat t pti isati n pr c ss, raining t a pling, as s r d 
t fail in t is d ain f applicati n. 

p ri nts n uns n alidati n data sugg st t at istri ut d s 
ar capa 1 f r ducing t pr 1 c pi it significant! . r suits 

s t at t classificati n accurac can aintain d il r ducing 
t f atur spac cardinalit a ut 5 %. n tic ig rit s ar d - 
nstrat d t seal 11 1 r larg -seal pr 1 s in atur 1 cti n. 



tr cti 

re t d lop t i I for atio c olog as roug t a out a tr 
dous flo of i for atio . o pa i s, orga isatio s a d i di iduals ar lit rail 
g tti g dro d i a flood of asur ts. 

att r cog itio , a d i particular patt r class! catio , off rs pro isi g 

a s to ards a or r d a of d ali g it i for atio . I ord r to 
i pro t proc ss of class! catio , atur 1 ctio a appli d to i cr as 

t t roug put of p rti t i for atio . It do s ot o 1 r due t a ou t of 

S. Cagnoni et al. (Eds.): EvoWorkshops 2000, LNCS 1803, pp. 77-86, 2000. 

© Springer- Verlag Berlin Heidelberg 2000 
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data tak i to accou t for a I for atio roc ssi g task, ut a also i pro 

t o rail proc ss accurac . 

Ho r pro isi g t s o s r atio s a , t r ar s rious c all g s 
to t: atur 1 ctio is k o to a -co pi t pro 1 . is fact 

ak s it tr 1 difficult to a i larg -seal , r al- orld do ai s. 

i c t arl studi s of atur 1 ctio t ods it s ti s, t 
otio of at “larg -seal ” actuall a s as c a g d. rst p ri ts 
d alt it rat r sm ll pro 1 s of so t s of f atur s. it t i cr as of 
CO putatio al po r, studi s of ^ r si tasks ( - 5 f atur s) ca 

possi 1 . oda , do ai s it or t a 5 f atur s ca said to trul 

c all gi g; su s qu tl , sue pro 1 s s all call d r I r si. 

r ar asicall to a s of carr i g out a atur 1 ctio proc ss: 
il r m sad r r m «[]• for r class of approac s r li s 

0 g ral statistical prop rti s of t pro 1 do ai a d a stracts fro par- 
ticular class! r od Is. rapp r t ods o t ot r a d opti is t p r- 

for a c of a CO cr t class! r. r as ilt r t ods t picall d li r or 

g ral r suits, rapp r t ods ar co sid r d to i Id a tt r class! catio 
p rfor a c for sp ci c tasks [2]. 

us, ai i g at a ig class! r accurac , rapp r t ods s to 
t or pro isi g ca didat s. ut t ast ass ss t ti for class! r p r- 
for a c 1 t sue approac s app ar i f asi 1 for r al- orld applicatio s at t 
rst gla c . Ho r, ad a c s i t r al of istri ut d rti cial I t llig c 
pro id a s to appl rapp r t ods to c all gi g r al- orld pro 1 s. 
folio i g pag s ill d scri a succ ssful applicatio of istri ut d 

tic Igorit s - a rapp r t od - to a r larg -seal pro 1 i atur 

1 ctio for a s st . irst, t g ral custo isatio of s for a- 

tur 1 ctio is outli d. It is folio d ad scriptio of our tailori g of s 

for r larg -seal atur 1 ctio . p ri tal s ctio d o strat s t 

us ful ss of t s tup. i all , t r suits ar su aris dad possi 1 t - 
sio s ar sugg st d. 

tic rit s t ct t r s 

s a pro d to capa 1 of d ali g it -co pi t pro 1 s i arious 

Ids. i dl ck ad la sk sugg st d t ir applicatio to atur 1 ctio for 

class! catio [3]. t r rapp r (a d ilt r) t ods a appli d i 

arious studi s [ ] . ut sp ciall i g fac d it r d a di g larg - 

seal tasks, s app ar to t st c oic du to t ir i r t parall lis 
a d o i al d a d for ackgrou d k o 1 dg . 

. si i p s 

1 t ir il sto pap r, i dl ck ad la sk us d a i pi to g rat 

pro isi gf atur su s ts. ai oft ir stud as to r due t f atur spac 
card! alit a d at t sa ti to k p t pr diet! accurac acc pta 1 ig . 
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o appl s, t tors arc rs “co r” t patt r s it a i ar ask 
t roug ic t classi r p rc i s t sa pi s - t f atur su s t. ’ ’ 
i dicat st prscada’’t ascofaf atur . c ro oso is 
o tai d tru cati g t ro s of t f atur ask. i dl ck ad la sk ’s 
od 1 as succ ssfull adopt dadr d a otrrs arc rs. 

CO o 1 appli d t ss fu ctio for a f atur su s t co sists of a 

CO i atio of classi r accurac a d su s t co pi it : 



I t is for ula, d ot s a accurac sti at for t i sta tiat d clas- 
si r, a d ( ) is a asur for t co pi it of t f atur s t - usuall t 

u r of utilis d f atur s. urt r or , i Ids t f atur spac cardi alit , 

ad is a pu is t factor to ig t ultipl o j cti s of t t ss fu c- 
tio . u r of f atur s us d a su s t is i t d d to 1 ad t algorit 

to r gio s of s all co pi it . 

. i is ri u s 

s tup as d scri d a o orks pr tt 11 for f atur spac s of co pa- 
rati 1 s all cardi alit . ut i g co fro t d it r larg -seal do- 

ai s, t ast ass ss t ti for f atur su s ts pro i its t is approac . 
u c t al plo d istri ut d s for atur tractio i ord r to 
stud ig r-ord r pro 1 s [4] . us d a icro rai istri ut d to 

acc 1 rat t aluatio proc ss, i Idi g up to li ar sp dups. 

Usi g sue CO tio al icro rai s, a f atur su s t is rat d s - 
di gittoar ot aluator. su s t’s pr dicti accurac is sti at d t r 
ru i g a CO pi t classi r. r suits ar t r tur d to a ast r od 
a d us d to guid t 

c tl , istri ut d r i I s r sugg st d i ord r to acc 1 rat 
t aluatio proc ss for classi r opti isatio s i furt r [5]. us of t is 
aluatio sc facilitat s for a stud of r larg seal do ai s i att r 
cog itio . is od 1 ca ciall appli d to atur 1 ctio . 



. s u i s 

ro toda ’s p rsp cti , past studi s appl i g tic Igorit s to atur 
1 ctio r li it d i arious a s: 

ajorit of r s arc rs i stigat d do ai s of rat r s all co pi it , 
t at is lo f atur s [3, 6- ] . 

ug unc t al. call t ir c ntri uti n rth r s arch atur 1 c- 

ti n o d lassificati si g tic Ig rith s , t in fact carr ut a at r 
tracti pr c ss. 
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tudi s orki g o larg seal ad r larg seal do ai s r li d o a rat r 
s all u roftrai i g data [9- ], ([ 2]). r- tti gt d ei s t us r 
quit lik 1 . alidatio as usuall ot r port d. 
o parati studi s of tie Igorit s a d ot r t ods r it r 

as d o s all-seal pro 1 s [ , 3, 4], or t eireu sta e s for s do 

ot app ar to eial [ 2] . 

su s qu t s etio s ill d seri a s to stud r larg -seal do- 
ai s i atur 1 etio il a oidi g t a o li itatio s. 

3 rsrrc is 

appl i g s to do ai s of larg eo pi it i atur 1 etio , t 
ajor e all g is t or 1 i g ti eo pi it of t proe ss. 



oug t ti r quir ts for t t ss aluatio ea dra atieall d - 
er as d irtu of istri ut d ( rtieal) s, t duratio of t ass ss t 
proe ss r ai s t t r at i g li itatio of t s are proe ss. t us 

as to i t good solutio si f r e el s as eo par d to ot r Ids of ap- 
plieatio . erueial trad off is to k p t aluatio proe ss as t si as 

r quir d - a d as eo eis as possi 1 . 

. I p c s s si 

I our stud , s ral asur ts r tak i ord r to eop up it t s 
r quir ts. ost of t s asur ts a r port d for ; it is t ir 

ear ful eo i atio t at faeilitat s for a suee ssful applieatio s of s to r 
larg -seal atur 1 etio . 

mi r rs[ 5] pro id a ffiei t a s to fore eo rg e of 

t s are proe ss. is is e ssar as t ti r quir ts to ass ss t 

i di iduals pr t t fro eo rgi g i a “ atural” a . 

ft r a aturi g p as of a out o - u dr d g ratio s, t si etio pr s- 

sur as i er as d i our s tup i ord r to 1 ad t algorit to a - e ssa- 

ril loeal - opti u . is as ae i d r duei g t utatio rat a d 

i er asi g erosso r rat a d i for atio i t re a g t i di iduals. 

i d seri s furt r d tails o t para t r s tti gs. 

I ord r to a e t i itialisatio proe ss, t ra do g rator as 
a ipulat d aeeordi g to a plieit i i lis i i s. I its usual i sta - 
tiatio , t i itial populatio oft is o tai d tossi g a fair eoi for 

r i di idual ad r f atur . I di iduals i r larg -seal applieati- 
o s ill r fl et t is e oie i t at t ill s tti d arou d t 5 % 1 1 

of eo pi it . algorit i g fore d to eo rg fast ould t us a 
a poor i itial eo rag of t s are spae . 
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Usi g a I itialisatio ias, t pro a ilit for a ’ ’ to occur i t i itial 
populatio is c os accordi g to 

( ) = = + 

o 

( ) = ( — — 



Its quatio s, t I itialisatio ias ra g s fro to - it r 

li arl or o tiall . o i d ot s t u r of i di iduals i t 
populatio adit u r of i di idual ( i o i ); ca 
adjust d usi g t fr para tru- 
st CO pi it of t i di iduals ca p ct d to r fl ct t is ias, t 

s arc spac co rag a larg d t is asur t. 

part fro t sta dard utatio op rator, s r s m i as 
appli d. roppi g t assu ptio of qual utatio pro a iliti s ad 

( d oti g t pro a ilit of ’i’ to c a g d i to ’j’) facilitat s for 

a ploratio of a road r ra g of t s arc spac . 

t di g t s t of usuall appli d t ss fu ctio as i troduc d i t 
for r s ctio , I mi I is m i s r appli d: 



op as t at t is asur t ould courag t algorit to 

plor r gio s of s all co pi it pr f ra 1 . 

0 a oid ositio al ias, i mi I r ss r- [ 6] as us d: i di iduals 

ar CO id tossi g a ias d coi for r g . sid s t is asur - 

t, ad appi gs r t st d to courag t productio of 

i di iduals i lo rad upp r ra g s of co pi it of t s arc spac . 

Iso sugg st d i [ 6] a d id 1 us d i atur 1 ctio studi s, rai- 

1 g t a pli g ( ) as tri d i ord r to acc 1 rat t p rfor a c 

ass ss t. id a is to ra do 1 s 1 ct a su s t of t classi r trai i g 

data for sti ati g t classi catio accurac of t su s ts. s du to 

t r -sa pli g t t ss asur c a g s i t coars of t s arc pro- 

c ss, i di iduals i rit d fro for r g ratio s a to r - aluat d. 

I t lit ratur , as r port d to sp d up t ss aluatio it out 

i troduci g too uc of ois i to t s arc proc ss 



p ri ts 

o alidat t us ful ss of t custo isatio s d scri d a o , t sts r co - 
duct d, tackli g a r larg seal task i atur 1 ctio . ti -co su i g 
p ri ts r CO duct d at t r a s arc t r for rti cial I t 1- 
lig c , Kais rslaut r , 



r a 
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up 

I our stud i stigat d a atur 1 ctio pro 1 for a s st . 

ur classi r as tailor d to cat goris a d ritt digits. 

s 1 ar i g data co sist d of i ar patt r s it 24 32 

pi Is, di id d i to t class s of qual cardi alit . data s t as us d 
s ral ti s for i t co u it , ost r c tl rakas a d urt 

[ 7] a d arad i [ ]. 

I ord r to i stigat t us ful ss of s i t is r larg -seal do ai , 

t 1 ar i g data as ra do 1 split i to a 5 % trai i gst, a3 %tstst, a d 

a 2 % alidatio s t at t gi i g of ac ru . sa pi s r or alls d 
to qual si i a pr proc ssi g st p, usi g a straig tfor ard scali g proc ss 

ic pr s r d t sp ct atio of t patt r s. 



ssi r o rat as t of f atur s, t 

to t trai i g s t usi g a - ar st- 

pro d t is od 1 to a ro ust, ffici tad 
-applicatio s [ 9] . ool a ista c usi g t 

a it-parall 1 appi goft patt r s. oug 

to acc 1 rat t classi catio proc ss, t ti 
su s t a ou t d to o - u dr d- ft s 
o illustrat t i pacts of t is gur : 
u dr d i di iduals a d t f atur spac 
i stigat d r , a austi s arc 
a out ars! 

uri g t opti isatio proc ss, t alidatio s t 
as us d to judg t g ralit of t r suits aft r t 
t ast r sourc r quir ts, co parati studi s it ot 
t ods r ot CO duct d. Ho r, t car ful p ri 
s parat alidatio s t facilitat s for a o j cti asur 
p rfor a c . 



t st s t as cat goris d accordi g 
ig our lassi r. ral studi s 
11-p rfor i g approac for 
op rator facilitat d for 
CO sid ra 1 ffort ad sp t 
r quir d to ass ss a si gl f atur 
orkstatio . 
to CO sist of o - 
76 , as i t pro 1 
ac i ould r quir 



CO ds o a U U 
i t populatio 
cardi alit to 
usi g a si gl 



as full k pt asid . It 
t r i at d. u to 
atur 1 ctio 
tal s tup it a 
t of t t od’s 



istri ut d rtical as ru o t irt U orkstatio s of 

t p - , -2 a d U .Its adapti load ala ci g pro id d 

for a ffici t us of t t rog ous ulti-us r iro t. or furt r 
d tails a out t co c pt of rtical s t r ad r is r f rr d to [5] a d [2 ]. 

populatio of o - u dr d i di iduals as ai tai d. o s 1 ct t pa- 
r ts of t g ratio , a co i d d t r i istic a d ra do strat g as 

us d: st i di iduals r copi d dir ctl i to t t g ratio , a d 

t r ai i g plac sit ati g pool r 11 d usi g a fortu 1 

as d o seal d t ss alu s. is scali g as do sue t at t diff r c s 
i classi catio accurac r a pli d. i i al t ss alu as add d to 
p r it for a s 1 ctio of i di iduals it lo pr dicti accurac . folio i g 
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quatio s o s o t t ss alu ( ) of a i di idual p is seal d, gi 

to t a rag t ss a d t a i al t ss i t populatio : 

. , ( ) - (2 - ) + t ss if 

' ' t ss t r is 

a 1 su aris s t ost i porta t para t r s tti gs i t coars 
of a t pical ru . d ot s t u r of g ratio s pass d, a d cross ias 
i dicat s t pro a ilit for t i o ial crosso r to c oos a g fro par t 

o for c ild o . 



g 


ara t r 


al 


ff ct 


- 5 


in itn ss 


A 


d rat s 1 cti n pr ssur 


( 1 rati 


utati n at 


2-3% 


f cus n pi rati n 




cr ss r at 


75% 


d rat c r s int racti n 




cr ss ias 


% 


1 g n int racti n 


5 - 75 
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.2 


incr as d s 1 cti n pr ssur 


(gr i g 


utati n at 
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pi rati n and c inati n 




cr ss r at 


% 






cr ss ias 


75% 


d rat g n int racti n 


75 - 


in itn ss 


.2 




( at ri g 


utati n at 


.5% 


f cus n aturing 




cr ss r at 


% 


incr as d c r s int racti n 




cr ss ias 


6 % 


incr as d g n int racti n 


- 


in itn ss 




ig s 1 cti n pr ssur 


(c rg c 


utati n at 


% 


nf rc c n rg nc 




cr ss r at 


% 


ig c r s int racti n 




cr ss ias 


5 % 


ig g n int racti n 



na ic p rat rs 



. su s 

si r i i pi rai i g t a pli g ( ) as us d 

a r s arc rs i t Id of atur 1 ctio to cop up it t o r 1- 
i g ti r quir ts. is t c iqu as as 11 tri d for t purpos s 
of t is stud . trai i g s ts a r -sa pi d at t % a d 3 % 1 1. 

i r s o s t olutio of t st t ss alu sot t st data, t at is 
t data us d i t opti isatio proc ss its If. trai i g s t as r -sa pi d 

i 3 % ratios, ot t at du to t is r -sa pli g t t ss fu ctio ari s fro 
g ratio to g ratio . 

s ca s it gur , o progr ss i t r s of t p rfor a c crit rio 

0 t t st data is o tai d. asur ts aki g us of t alidatio data 

tur d out to or disastrous. ois i troduc d to t s arc proc ss 

so d to too ast for t is ig -di sio al datas t i co ju ctio it t 

ig u r of trai i g i sta c s. us, as ot furt rl co sid r d 

1 t is stud . 
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0 20 40 60 80 100 120 140 160 180 

Generation number 

i . . st fitn ss alu s using raining t a pling 



rc p c r Usi g t sta dard op rators, t as o s r d 

to trac i di iduals ar t 5 % 1 1 of co pi it pr f ra 1 . is a la- 

d to t p culiariti s of t pro 1 do ai , sue as ig o - o oto ousit 
of t t ss fu ctio , ig i s siti it of t lassi r to c a g s i f a- 

tur spac cardi alit , t li it d aturi g p as ads arp p rfor a c p aks 
i ra g s of lo co pi it . 

o g t asur ts sugg st d i t for r s ctio , as c ro ous u- 
tatio tur d out to t ost ff cti as to road t a i d r gio . 

i corporatio of a pu is t factor i to t t ss fu ctio alo as ot 

capa 1 of guidi g t algorit to ot r r gio s of i t r st. us of 

appi gs i suppl t of so of t crosso r op ratio s as o s r d to 
1 ad to pr atur co rg c : algorit got stuck i su opti al r gio s 

it pr dicti accurac lo t full f atur s t. 

c i i urt r p ri ts usi g t full trai i g s t for accurac sti a- 
tio r CO duct d. r suits s o t at s seal 11 to do ai s of larg 
CO pi it i atur 1 ctio : a s of d a ic op rators, t co - 

rg d it i a out t o- u dr d g ratio s. u r of utilis d f atur s 

could r due d a out 5 % il pr s r i g t pr dicti accurac of t 
classi r. 

i r2sost d lop t of t p rfor a c o t us alidatio 
patt r s: ft r r ac i g a c rtai d gr of aturit , t calculat d f atur su - 
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s ts d li r classi catio accuraci s co para 1 to or tt r t a t full 

f atur s t. is r arka 1 r suit could r produc d r lia 1 i t co s cu- 
ti ru s. 




i . . alidati n p rf r anc 



r t k 

is r port ai d at i stigati g t us ful ss of tic Igorit s for r 

larg -seal atur 1 ctio . ral partiall air ad r port d asur ts 
r CO i d i ord r to cop it t co pi pro 1 

o li itatio s of s for r larg -seal atur 1 ctio pro 1 s r 
osr dad astoo rco t r sugg st d. p ri ts s o d t at 

s seal 11 to do ai s of larg co pi it i atur 1 ctio . 

id 1 appli d appro i atio t od, rai i g t a pli g, as o s r- 

d to fail grossl i our applicatio . olio i g t asic id a of t is approac , 
or suita 1 custo isatio s s ould tri d i futur studi s. I st ad of ra - 
do 1 r -sa pli g t trai i g s t, a rotot p 1 ctio proc ss a carri d 

out si ulta ousl to atur 1 ctio . arad i appli d t is id a succ ssfull 

to s all-seal att r cog itio pro 1 s [ ]. is co i atio app ars to 
pro isi g for r larg -seal tasks, as 11. 
utur ork ill ai at a stud of furt r atur 1 ctio t ods i r 
larg -seal do ai s. pro 1 of atur r atio ill addr ss d i ord r 
to act s arc proc ss i furt r. 
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c . his pap r pr s nts an in stigati n int th classificati n f 
a difficult data s t c ntaining larg intra-class aria ilit ut 1 int r- 
class aria ilit . tandard classifi rs ar ak and fail t achi satis- 
fact r r suits h r, it is pr p s d that a c m inati n f such ak 
classifi rs can inipr rail p rf rmanc . h pap r als intr due s 

an 1 luti nar appr ach t fu rul g n rati n f r classificati n 
pr 1 ms. 



I tr cti 

his pap r d scrib s a s ri s of p ri nts in tac ling a difficult class! cation 

probl . h data consists of arious b ans and s ds, a pi s of hich ar 

sho n in gur . Ithough so of th obj cts ar larg r than oth rs ( .g. 
al onds co par d to 1 ntils) ar int r st d in classif ing th bas d on 
th ir shap alon ithout using infor ation about th ir si . his corr sponds 
to th situation h r th distanc b t n th obj cts and th ca ra is not 
d, and so th ir appar nt i ag d si s ould ar . h difficult of th tas 

li s in th r lati 1 s all int r-class diff r nc in shap and th high intra- 

class diff r nc s. In oth r ords, all th obj cts loo si ilar, app aring roughl 
lliptical. Ithough th shap s of so obj cts ( .g. al onds) ar fairl consist nt 
oth rs ar consid rabl ( .g. corn rn Is). 

h basis for classif ing th obj cts ill b a s t of shap prop rti s asur d 
fro th ir silhou tt s. inc si infor ation is to b discard d th prop rti s 
n d to b in ariant to scaling, i is , in arianc to position and ori ntation 

S. Cagnoni et al. (Eds.): EvoWorkshops 2000, LNCS 1803, pp. 87-96, 2000. 
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chang s is n c ssar . urth r or , it a b us ful to includ in arianc to ad- 
ditional transfer ations of th shap . or instanc , if th d t r ining shap 

factor of a class is its si ilarit to an Hips th n th asp ct ratio a b irr 1- 

ant. h CO put r ision lit ratur pro id s a ari t of shap asur s [ ]. 
s 1 ction of th s , in co bination ith so n shap prop rti s d lop d 

b osin [2], ha b n appli d to g n rat a s t of 7 asur nts of ach 

sa pi . h can b di id d into subgroups according to th ir prop rti s and/or 
algorith s: 

s - four attribut s in ariant to rotation, translation, and scaling 
(in ariant und r si ilarit transfer ations). 

s 2 - thr attribut s in ariant to rotation, translation, scaling, 
and s (in ariant und r affin transfer ations). 

s r - four standard attribut s - cc ntricit , circularit , co pactn ss, 

and con it (in ariant und r si ilarit transfer ations). 

r - thr asur nts of llipticit , r ctangularit , and trian- 
gularit (in ariant und r affin transfer ations, si ilarit transfer ations 
and str tching along th a s, and affin transfer ations r sp cti 1 ). 

r 2- thr alt rnati asur nts of llipticit , r ctangularit , 
and triangularit . 



#• • 










• • »•••'••% 

i . . ampl s f data; r sc ntain: ( ) aim nds, (2) chickp as, (3) c ff ans, 

(4) 1 ntils, (5) p anuts, (6) c rn k rn Is, (7) pumpkin s ds, ( ) raisins, ( ) sunfl r 
s ds. 



In this pap r ha in stigat d four diff r nt classi cation t chniqu s, 
and CO bin d th in an att pt to i pro o rail classi cation. o i pi - 
ntations of d cision tr s ar us d. h rst thod is th 11 no n 4.5 
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achin 1 anting thod d lop db uinlan [3]. 4.5 indue s tr s that par- 

tition f atur spac into qui al nc class s using a is-parall 1 h p rplan s ( .g. 
in 2 this ould consist of hori ontal and rtical lin s) . h s cond approach 

is [4] hich is a g n ralisation in that, rath r than eh c ing th alu of a 

singl attribut at ach nod , it t sts a lin ar co bination of attribut s. atur 
spac is cons qu ntl partition d b obliqu h p rplan s. h third thod is an 

ns bl of n ural n t or s that ar train d on diff r nt f atur s patt rns, in- 
cluding on n t or that incorporat s all patt rns. inall , ha also ol d 

fu rul s using olutionar progra ing. 

2 ri f r i f cti ist cisi r 

ssi c ti 

ural n t or s ar conn ctionist s st s that ar id 1 us d for r gr ssion and 

function appro i ation. h co pris input nod s, hich ar us d to pro id 

training data to th n t or and output nod s us d to r pr s nt th d p nd nt 
ariabl s. Input- utput apping is achi d b adjusting conn ction ights 
b t n th input and output nod s, but or usuall , through an int r diat 
la r of nod s. his charact ristic can b odi d for classi cation probl s b 
sp cif ing d sirabl n t or outputs to b binar alu s. ural n t or s ar 

r liabl us d in classi cation of co pi data. 

u classi cation is a rul -bas d approach in hich I - H rul s ar us d 
to cat goris data, h rul s r lat g n rails d or i pr cis groupings of input 
data, and th d cision of a gi n rul r pr s nts a d gr of b longing to a gi n 
output class, his t p of classi r is particular! us ful h n it is n c ssar 
to pro id int rpr tabilit to th classi cation s st in th for of linguistic 
rul s. Ho r, th proc ss of cr ating classi cation rul s is oft n difficult and 

ti -consu ing. ral studi s ha att pt d to cop ith this probl using 

1 arning algorith s, and in particular th us of n ural n t or s. 

cision tr s ar a 11 stablish d t chniqu bas d on structuring th data, 
according to infor ation th or , into utuall clusi crisp r gions. lassi - 
cation is g n rail p rfor d b starting at th tr ’s root nod and eh c ing 
th alu of a singl attribut , and d p nding on its alu s th appropriat lin 
is folio d. his proc ss is r p at d at succ ding nod s until a 1 af is r ach d 
and a classi cation is assign d. n rating th opti al d cision tr is g n r- 
all -hard, and th r for a sub-opti al tr is indue d inst ad, using gr d 
or rando is d hill cli bing algorith s for instanc . h r suiting tr is oft n 
prun d; subtr s ar r plac d b 1 af nod s if this r due s th p ct d rror 
rat s. his r suits in pot ntiall s all r and or accurat tr s. 

3 ti r ssi c ti 

olutionar t chniqu s ha not b n pr iousl appli d to fu classi cation 
probl s. h clos st r lat d or studi s us g n tic algorith s to opti is 
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fu classi cation rul s and th ir b rship functions, n disad antag ith 
this approach is that it is in ariabl n c ssar to pr -sp cif th structur of th 
rul s, hich oft n r suits in sub-opti al classi cation. In this pap r, ha 
propos dan t chniqu in hich fu classi cation rul s of arbitrar si 

and structur can b g n rat d using g n tic progra ing. his is d sirabl 
for CO pi probl s, ith larg nu b rs of input f atur s, for hich it is not 
f asibl to for ulat th structur of rul s anuall . urth r or , ith such 
larg nu b rs of f atur s it is usuall th cas that c rtain f atur s ar not 
signi cant in classi cation of diff r nt output class s. H nc , in this cas , 
can sa that g n tic progra ing is us d for unconstrain d rul disco r and 
opti isation. 

n tic progra ing is an olutionar t chniqu in principl to Holland’s 
g n tic algorith s. h ain diff r nc s ar , ( ) th structur of a g n tic pro- 
gra is a tr , (2) th nod s of th tr s ar functions (or t r inals), hich 

nabl s th tr s to int rpr t d as progra s and (3) th si of ach tr in a 

population is ariabl , unli ost g n tic algorith s h r all indi iduals ar 
th sa si . th r is , standard op rators applicabl to g n tic algorith s 
ar us d in g n tic progra ing. 

In this stud , th nor alis d input spac as partion d into thr fu 
b rship functions, r and s [5]. t , Z, b th fu 

b rship functions. assu d si pi rul constructs co pris d of t o 
inputs and on output, h non-t r inal nod s of th tr s r pr s nt th s 

si pi rul s, hich ar co bin d to for th co pi classi cation rul . ach 

si pi rul is aluat d b atching its inputs against th fu ant c d nts 
and th output is obtain d using an op rator, na 1 I . hus, a nod 

pr ss d as Z ( , ) is int rpr t d as: 

I ( ) H I ( ) 

hr is th d gr of b longing of to th fu b rship function Z. 

his t p of rul construct is pr f rabl to dir ct co bination of th input pa- 
ra t rs b caus it assists in int rpr tabilit of th classi cation s st .hr 
ar nin diff r nt fu rul s hich can b for d fro th co bination of th 
thr b rship functions. 

h stud us d sourc cod (lilgp) d lop d at ichigan tat ni- 

rsit [6]. his as us d to ol a s t of co pi fu classi cation rul s, 

on for ach output class of data, h function s t is co pris d of th si pi 
rul s d scrib d abo , hil th t r inal s t co pris d rando constants and 

CO on arith tic op rators. h tn ss of th tr s on th ir aluat d out- 

puts r d t r in d against targ ts of . for th corr ct data class and . 
oth r is . s an a pi , th folio ing is a s all portion of a co pi rul : 

(ZP (NP a b) (ZN d c) ) 

It can b int rpr t d as: 

IdisZ cis Htp=I( ) 

I a is b is H t p2 = I ( ) 
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I t p is Z t p2 is H 

I ( t p t p2) 

ti c s 

caus n ural n t or s ar train d on li it d sa pi s ts of r pr s ntati 
data th r ar al a s signi cant rrors in th g n ralisation of co pi func- 
tions. n a to o rco this probl is to train ultipl n ural n t or s on 

ind p nd nt data s ts and th n us oting sch s to d t r in an o rail clas- 

si cation [7]. h t o popular sch s ar co onl no n as ns bl and 

odular n t or s. In ns bl s or co itt s r dundant n t or s ar train d 

to carr out th sa tas , ith oting sch s b ing appli d to d t r in an 

o rail class! cation, n th oth r hand, it is pointl ss to train id ntical n ural 

n t or s and consid ration is thus oft n gi n to using diff r nt topologi s, data 

s ts or training algorith s. In odular n t or s th classi cation probl is 

d CO pos d into subtas s. or a pi , n ural n t or s ar train d to r spond 

to on class or a group of class s. 

h output of a classi cation n t or can b us d to indicat th d gr to 
hich th input f atur s ar atch d to th diff r nt class s. hr for a si pi 
approach to n t or co bination is to su th acti ation 1 Is of corr sponding 

output nod s. r n nt to this sch is to seal th output 1 Is ithin a 

n t or such that th su to on . his alio s th contributions across n t or s 

to b or CO parabl . 

n alt rnati approach is bas d on th confusion atri , a tabl containing 
ntri s , hich indicat th fr qu nc that data sa pi s fro class r 

lab 1 d as class . uch a tabl is us ful for anal sing th p rfor anc of a 

classi r. ur approach is bas d on th classi cation accuraci s for ach class 

gi n b 



h contributions of ach classi r ar ight d b th s p ct d accuraci s. 

n a pi of a confusion atri is sho n in abl . It can b s n that th 
classi r is capabl of consist ntl corr ctl classif ing all instanc s of class , 
but onl 75% of class2. 

p ri ts 

h p ri nts co par d th p rfor anc of four classi rs on th shap clas- 
si cation probl . h training and t sting data both consist of 3 sa pi s 
ach containing s nt n continuous attribut s of shap . olio ing standard 
practic th data as nor alis d prior to pr s ntation to th n ural and fu 
classi rs, h r as this as not n c ssar for th d cision tr s. 

h rst s t of p ri nts cone ntrat d on aluating ach indi idual clas- 
si r. a pi s of th ir outputs ar sho n in igur s 2 to 4. h ir p rfor anc s 
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ar sho n in abl 2. h botto thr ntri s corr spond to alt rnati ans 

to d ri ing a classi cation bas d on n ural output nod acti ation 1 Is. h s 
ar standard inn r-ta -all ( ), and using th tod cision tr s( and 

4.5). It is of int r st to not that th si pi inn r-ta -all p rfor s b tt r 
than th or co pi d cision tr s in co bining th outputs of th n ural 
n t or s. 

h s cond s t of p ri nts co par d arious oting sch s appli d to 
n ural n t or ns bl s. h n ural n t or s r train d on th sub- 
groups of shap prop rti s, and an additional n ural n t or as train d on th 

CO bin d s nt n prop rti s. h oting sch s ith r s 1 ct onl th a i- 

u acti ation 1 1 ithin a n t or ( inn r-ta -all), or alt rnati 1 all th 

acti ations 1 Is ar us d. hi Is a rst b seal d, and ight d b th 

confusion atri , and ar th n su do r th n t or s. hr suit in abl 3 

sho s that b tt r p rfor anc is achi d h n th n t or ns bl includ s 

th n t or train d ith all prop rti s. urth r or , th us of th confusion 

atri g n rail i pro s p rfor anc . Ho r, th odular approach s pro- 
id littl i pro nt o r th basic singl n t or clasi cation r suit. 

h nal s t of p ri nts in stigat d h bridisation of thr classi rs, 

na 1 , n ural n t or , fu - and 4.5, using th confusion atri thod. 

his do s not r 1 on pot ntiall inco nsurat li lihood alu s that a 
b produc d b diff r nt classi rs. s fro abl 4 that furth r gains in 
p rfor anc ha b n achi d, indicating that th n ural n t or and d cision 

tr pro id us ful co pli ntar infor ation. 

6 c si s 

his pap r has pr s nt d a co parison of diff r nt classi cation t chniqu s on a 

difficult shap anal sis probl . i ilar to oth r r ports in th lit ratur [ , 9] 

it has b n sho n th r is no signi cant diff r nc s b t n th indi idual t ch- 
niqu s on our classi cation probl . Ho r, ha sho n that i pro nts 

can b achi d through diff r nt co binations of th s t chniqu s [ ] . 
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i . . n ampl f th partiti ning f f atur spac f r th an classilicati n task 
th d cisi n tr using nl t pr p rti s. h h ri ntal a is pr p rt is 

llipti cit , and th rtical a is pr p rt is triangularit . 
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Rule 4: 

property 16 <= 0.225411 
-> class 6 

Rule 7 : 

property 1 > 63.7441 
property 1 <= 65.4694 
property 4 > 0.161205 
property 4 <= 0.179735 
property 11 <= 0.967232 
property 13 > 0.710997 
property 15 <= 0.061688 
property 16 > 0.225411 
property 16 <= 0.329981 
-> class 8 

Rule 8 : 

property 1 > 65.4694 
property 4 <= 0.179735 
property 15 <= 0.061688 
-> class 6 



i . . t pical s t f ml s g n rat d 4.5 



(ZP (NP b 

(PN (- (PN a 0.83860) 

(NP (ZP (- (- (ZN d c) 

(NZ -0.02504 d)) 
(PZ a f)) h) 

(PZ e h))) g)) 

(ZP a 

(ZP (- (ZN d c) 

(NP d 

(ZN d c))) 

(ZP (NP f b) 

(ZN d c))))) 



i . . p rti n f a c mpl fu - rul in pr fi n tati n. pp r cas 1 tt rs 

d n t fu rul s, and 1 r cas 1 tt rs d n t i nput shap pr p rt alu s. 
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n approach to g n rat fu clasi cation rul s using g n tic progra - 
ing, on oth r hand, has not d onstrat d satisfactor r suit, his a b du 

to th li it d partitioning of th fu input spac into onl thr b r- 

ship functions i pi nt d in th stud and furth r or ill in stigat th 

pot ntial of this thod. 
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Abstract Modern largeftcale fifunctional genomicsfl projects are inconfl 
ceivable without the automated processing and computerfhided analysis 
of imagesflThe project we are engaged in is aimed at the construction 
of heuristic models of segment determination in the fruit fly s 

s flThe current emphasis in our work is the automated transfl 
formation of gene expression data in confocally scanned images into an 
electronic database of expressionfl We have developed and tested profl 
grams which use genetic algorithms for the elastic deformation of such 
imagesflln additionfl genetic algorithms and the simplex methodflboth 
separately and in concertflwere used for experimental determination of 
s embryonic curvilinear coordinatesflComparative tests demonfl 
strate that the hybrid approach performs bestflThe intrinsic curvilinear 
coordinates of the embryo found by our optimization procedures appear 
to be well approximated by lines of isoconcentration of a known morfl 
phogenfl Bicoidfl 



Introduction 

Computer Aided Analysis of Biological Images 

The ongoing revolution in molecular genetics has progressed from the large scale 
automated characterization of genomic sequence to the characterization of the 
biological function of the genome These investigations mark the beginning of 
the era of functional genomics A key feature of genomic scale approaches 
is the automated treatment of large amounts of data Both current and future 
work in the eld is impossible without the automated processing and computer 
aided analysis of images in connection with updating interactive electronic image 
databases 

A key aspect of such processing involves the segmentation of individual im 
ages and the registration of serial images Many problems involving the recog 
nition classi cation segmentation and registration of images can be formulated 
as optimization problems These optimization problems are typically di cult 

S. Cagnoni et al. (Eds.): EvoWorkshops 2000, LNCS 1803, pp. 97-106, 2000. 
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involving multiple minima and a complex search space topologyflContemporary 
approaches based on evolutionary computations are a promising avenue for the 
solution of such problems fEvoIASPflfifil 

Here we describe a new method for the determination of intrinsic biological 
coordinates in embryos of the fruit fly Drosophila melanogaster by means of 
genetic algorithms flGAsfEGAsflthe simplex methodfland a hybrid of both were 
applied to the problemfland we find that hybrid methods perform the bestflOur 
results indicate that these coordinates may be determined by a morphogenetic 
gradient of the protein BicoidHa result of some biological interestfl 



Stripe Straightening Search of Intrinsic Coordinates of Early 
Embryo 

Early in the development the fruit fly embryo is shaped roughly like a hollow 
prolate ellipsoidfl composed of a shell of nuclei which are not separated by cell 
membranesfl Deviations from the elipsoidal shape reveal the future polarity of 
the animalft bodyflThe more pointed end on the long axis makes anterior flheadfl 
structuresfland the rounder end posterior fltailfl structuresflFrom a lateral fbidefl 
perspectiveflone long edge of the embryo is flat and will will make dorsal filbackflfl 
structuresflwhile the other long edge is rounded and makes ventral filundersidefifl 
structuresflln this paper we follow the standard biological convention and show 
embryos with anterior to the left and fif a lateral viewfl dorsal up fFigflflfE 
Figfl.fl shows that so called pairfhule stripes flearly markers of the future 
segmental pattern fllfH are not parallel and straightfl but have a crescentflike 
formflThe curvature of the stripes is highest at the terminifland minimal at the 
central partflEach stripe specifies an anteriorfjaosterior flAfPfllocationfland these 
stripes can be regarded as contours in an intrinsic coordinate system that is being 
created by the embryo itselffl Another set of embryonic determinants exists for 
the dorsofkentral flDflVfl axisfllf the image is smoothly transformed such that the 
curvilinear coordinates are plotted orthogonallyflthe stripes appear straightflso 
the determination of these coordinates can be viewed as a flstripe straighteningfl 
procedureflOur task is to understand and characterize this curvilinear coordinate 
system as it relates to the AfP axisfl 

Two coupled objectives of this study arefl 

flfl To characterize the intrinsic embryonic curvilinear coordinatesfl 
flfl To use carefully characterized and tested computational procedures for the 
purpose of automatically processing large numbers of imagesfl 



Methods and Approaches 

The work reported here is part of a large scale project to construct a model of 
segment determination in the fruit fly D melanogaster based on coarsegrained 
chemical kinetic equations fflfHThe acquisition and mapping of gene expression 
data at a heretofore unprecedented level of precision is an integral part of this 
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Fig Image of early fblastoderm stagefl fly embryo with crescentflike stripesfl in 
Cartesian physical coordinatesfl This is a confocally scanned image of an embryo 
stained by indirect fluorescence flimmunostaining with polyclonal antisera against the 
segmentation proteinflflEach small dot is an individual nucleusfl 



projectflThe current emphasis in our work is on the automated transformation 
of gene expression data in confocally scanned images into an electronic database 
of expressionfl 



Images of Drosophila Genes Expression 

Transformations of embryonic coordinates begin with data expressed in terms 
of the average fluorescence level flproportional to gene expression levelfl of each 
nucleusfl where segmentation proteins exert their biological functionflThis data 
was obtained as followsfl 

Antibodies for flfl protein products of segmentation genes were raised and over 
flflfl images were prepared and scanned fllfflThese images were computationally 
treated by means of the Khoros package fHfllEmbryos were rotated and cropped 
automatically such that the physical long axis of the embryo was parallel with 
the X axis and the short physical axis with the y axisfl 

Nextflthe images were segmented fllflKosman et alflin preparationffl About 
flflflfl segmented and identified nuclei are obtained from each imageffEach nucleus 
is labeled numericallyfl and the x and y coordinates of its centroid are foundfl 
together with the average fluorescence level over that nucleusfl The segmented 
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data takes the form of tables in ASCII text formatflThe result is the conversion of 
an image to a set of numerical data which is then suitable for further processingfl 



Stripe straightening algorithm 

In Figflfl the crescentflike pairftule stripes of an embryo in near saggital projecfl 
tion are shownflWe assume that the center of a pairftule stripe follows a curve 
of constant AfP positionflThe origin of the image coordinate system is at the 
top leftflwith image coordinates for width w increasing to the right and height 
h increasing downfl 

Our goal here is to find the true AfP and DflV coordinates on the imageflWe 
approximate the true coordinate system by a Taylor series as followsflWe denote 
the true AfP coordinate by fl and the true DflV coordinate by ^We pick the 
origin of flfl pi and the origin of new image coordinates ffc yf\. so that they are 
the same and as close as possiblefl 

We note that there is an AfP position at which a stripe is exactly vertical 
on its whole lengthflThe center of that stripe defines fl fl flfl which is the ^fhxisfl 
Each pairfrule stripe other than the one at fl fl fl is curvedfland we imagine the 
fl axis to intersect each of the stripes at the point where it is exactly verticalfl 
Now we pick new image coordinates x and y such that they have the same origin 
and orientation as the flfl coordinatesflthat is 

a; fl w — w y fl —h — h flflfl 

We now turn our attention to flflFor nowflwe can assume that ^fl yflEven 
if we donfl do thatfltwo important things will be true about the relationship 
between fh yfland flfl pHflflfl The y and ^axes are coextensivefland flflfl The loci 
§. fl const are orthogonal to the y and §. axes as they cross y fl ^ fl flfl Both of 
these important points follow from the existence of the vertical stripeflWe would 
like to write fl in terms of x and j/flso that 

fl fl /flr yfl flflfl 

We expand in a Taylor series to third order around the originflThat gives 

flfl /flfl flfl fl — X & 

fl flfl flfl — a: fl flfl flfl xy fl 

fl flfl flfl — a: fl flfl flfl a: y fl 

fl flfl flfl — y 

flflfl 

Now consider the terms and what they meanfl/flfl flflfl fl by deflnitionflWe 
picked flfl §0. such that at the origin — flfl and — fl flflFor pure y terms we can 
say more than thatflThe fact that the y and the ^ axes are coextensive means 
that /flfl yflfl fl yfland so — fl — fl fl as wellflThus far we have shown 
that five of the ten terms of the Taylor expansion vanishfl 



— y 
flfl flfl — 
flfl flfl 
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The unit vector e in the fl direction is proportional to — flso — measures 
the change in length of e as we move along the x axisflThis means that — fl flfl 

Now consider flThis term can be thought of as the rate of change in size of 

the unit vector e fl — along the yfkxisflAlong the ?/fhxis where a: fl fl fl flfl 
— fl fl yflso that derivatives of this quantity with respect to y vanishfland 
hence this term of the series vanishesfl This has eliminated all but three terms 
from the seriesflso now we write the first order model of image transformation 
as 

fl fl a; fl Axy fl Bx y fl Cx flflfl 

All of these terms have a clear interpretationflThe xy term is the main onefl 
it gives quadratic DflV curvature that increases with distance from the a;fhxisfl 
The X y term gives residual DflV asymmetry and the x term gives residual AflP 
asymmetryfl Lastlyfl if one expresses the above equation in terms of w and /rfl 
expansion will bring back lower order terms in h and w when expanding 

fl fl w — ao fl Aflat! — u> flfl— /i — fl fl BSw — ac fl fl— /i — /a flfl Cflu; — au fl flflfl 
in terms of w and /ifl 

We tested this flftt order model and found that in more then half of cases 
it is insufl cient for straightening stripesflWe expanded the model empiricallyfl 
with the result that an empirical extension of the flftt order model is given by 

fl fl Aflat! — at! flfl— /i — fl fl BSw — ac fl fl— — /i flfl 

Cfltt! -w SSr~h-h S S DSw - w flfl-/i -h S ™ 

We can treat of these additional fourth order members as followsflCx a/ is a 
correction term for parabolic splayflwhile Dxy serves to correct DflV asymmetryfl 
In generalflthe situation is typical of a polynomial approximation problemfl there 
is one polynomial that is best but there are a number of distinct ones that can 
approximate it very wellfl 

Preliminary calculations have shown that the best outcome is achieved with 
an independent deformation of the anterior and posterior half of an embryoflin 
summary it requires the determination of fl parameters of a deformation plus an 
evaluation of values w Sh Sh S 



Genetic Algorithms Technique Simplex Method and Their 
Hybrid 

GA Search The optimization problem of finding the coefl cient values for 
proper elastic transformations was initially implemented with GAsflWe have 
reduced the problem to the determination of factors ASBSC and D of equation 
flfl 

We use the following cost functionflEach embryofl image under consideration 
was subdivided into a series of longitudinal stripsflThen each strip is subdivided 
into boxes and the mean value of the product SEVEN SKIPPED proteinfl is 
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calculated for each boxflEach row of means gives the local profile of even skipped 
gene expression along each stripflThe cost function is computed by comparing 
each profile and summing the squares of diflerences between the stripsflThe task 
of the GA is to minimize this cost functionfl 

Following the classical GA algorithmflthe program generates a population of 
floatingfjjoint chromosomesfl Initial chromosomes are randomly generatedfl Affl 
ter that the program evaluates every chromosome as described abovefl. thenfl 
according to the truncation strategyflthe average score is calculatedfl Copies of 
chromosomes with above average scores replace all chromosomes with a score 
less than averagefl 

On the next step a predetermined proportion of the chromosome population 
undergoes mutationflso that one of the coefl cients gets a small incrementflThis 
cycle is repeatedflall chromosomes are consecutively evaluatedflthe average score 
is calculated and the winnersfloflspring substitutes for the losers in the process 
of reproductionfluntil an accepted level of stripe straightening is achievedfl 



Simplex Search We also solved the optimization problem by the downhill 
simplex method in multidimensions of Nelder and Mead fHfflThe method requires 
only function evaluationflnot derivativesflThis is an important speed advantage 
over gradient methodsfl since calculation of the gradients requires many more 
evaluations of the cost functionfl 

A simplex is the geometric figure in N dimensions of A^ fl fl vertices and all 
their interconnecting line segmentsflThe NelderfMead method starts with such 
a set of Al fl fl points defining an initial simplexflThe downhill simplex method 
operates by moving the point of the simplex where the function is largest through 
the opposite face of the simplex to a lower pointfland so on until it reaches the 
vicinity of an extremumfl 



Hybrid Procedure Initial experience indicated that that the simplex method 
is fast but does not give high quality answersflwhile GAs give excellent answers 
but are slowflWe noted that both multiple simplex runs and GA search perform 
numerous evaluations for many random points in search spacefllf we use small 
increments as mutations we will perform practically the same search by using 
GAs or the simplex methodfllf soflwe could use a set of chromosomes from 
the GA technique as a starting simplex for NelderfMead optimizationfl In the 
hybrid algorithmflwe implement a simple evolution strategy with floatingfjooint 
chromosomes with small mutational incrementsflSelection and reproduction are 
performed as described aboveflln additionflfrom the very beginning the program 
links pointers to mutant oflspring so as to achieve complete lists of Alfl fl pointers 
on Al fl fl relativesfl 

These flclans of mutantsfl are ready for simplex procedurefl Following the 
completion of the first list of Al fl fl pointers the program starts to perform not 
only mutationflselectionfl reproduction proceduresflbut also the simplex procefl. 
dure for the lowest scoring members of flcompletefl clansflThe more clans achieve 
completion the more species undergo simplex procedureflln summaryflGAs must 
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provide search of global optima together with local onesflwhile simplex provides 
fast downhill movingfl 

Results and Discussion 

Search Space Features for Stripe Straightening Problem 

The above described task of image elastic deformations turned out to be a diffl 
flcult numerical problemflThis is caused first of all by the unusual geometry of 
search spaceflFigflfl gives a picture of its features through crossections of the 
search space for one typical embryo under considerationfl 




Fig Search space features for one of crossections M fi Dfl for typical imagefl This 
is surface plot where vertical BZB. axis is evaluation onefl while X and Y are A and D 
coefl cients of expression flfl 



As we can see this cross section includes two groovesflone of which is deeper 
than anotherflThe sharp rectangular walls in FigfH are caused by penalty confl 
ditionsflThe omission of the penalties gives a smoother surface with one groovefl 
which corresponds to the deeper groove flnot shownffiln turnfl penalties are abfl 
solutely needed to avoid highly nonlinear folding of an image instead of smooth 
deformationsfl 

The bottom of both grooves have several local minimaflAs a result the simplex 
search gives in serial searches tens of such local extremaflGA search is more 
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eflective and it finds the best local extremum on the bottom of the deepest 
grooveflHowever to jump from the shallow to the deeper groove is still a difl cult 
task for GA search as wellflTo overcome this we use large population sizes or a 
series of runs to achieve the best solutionfl 



The Results of Stripe Straightening 

After completion of the stripe straightening procedure with fifi coefl cients ftu 
and two sets of Afi i?fi Cfi D and h fi /i fi for about two hundred images from 
the stages when all seven crescent stripes are visible we could compare found 
coefl dent setsfi These coefl dent sets show considerable diversityfi so that we 
fail to elucidate a general formula of appropriate elastic deformations to achieve 
satisfactory stripe straighteningflHoweverfithe resulting transformation of coorfl 
dinates are very similar for most of the imagesfi Typical example is shown in 
Figffifi 




Fig Typical example of curvilinear coordinates found by our optimization procefl 
duresfl 



On the contraryficomparison of coordinate curves for anterior and posterior 
halves of embryos reveal small but quite evident difierences fiCfflFigfH and Figffififi 
A biological subject of interest is the source of the pairftule stripesficurvaturefi 
It is known that in Drosophila segmentation the maternally expressed protein 
BICOID forms an anteriorf|)osterior morphogenetic gradient in the egg which 
controls all following segmentation events fllfilit is interesting that contour lines 
of a fiDfboncentration map of the BICOID gradient closely coincide with curvifl 
linear coordinates determined by our methodflThe full biological implications of 
this observation will be reported elsewherefi 



Experimental Determination of Drosophila Embryonic Coordinates 105 



E ectivity and Cost of GAs Simplex and Hybrid Techniques 

In a table fl the results of the comparative tests on elastic transformation of 
a typical image by means of the simplexftnethodfl G Asfl and GAs with simplex 
flthe hybrid techniquefl are presentedflTo find parameters of optimization for a 
simplex and GAs giving the most effective optimizationfl careful tuning of the 
limits in variation of a mutational increment and the range of variability of 
the initial population was carried outflThe hybrid method was tested at the 
same values of parametersflas GA techniqueflThe result of testing was compared 
according to the time required for calculation and according to the standard 
deviation of the resultflOn each tested procedure flflfl independent runs were 
carried outfl Inspection of the table reveals that the simplexfinethod is fastestfl 
but also the least preciseflThe GA technique is the most preciseflbut also requires 
flfl times as much computingfl Our results indicate that the hybrid technique is 
approximately twice faster than GAs at the same accuracyfl 



Table Comparison of effectiveness of three approaches on calculation time fin a 
summarized amount of evaluationsfl and on a divergence fin a standard deviation valfl 
uesfl 



Method 


Time in evaluations 


Standard Deviation 


Simplex Method 


flflflflfl 


flflflflfflflfl 


GA Technique 


flflflflflfl 


flflflffiflfl 


Hybrid Technique 


flflflflflfl 


flflflfflflfl 



The problems we encountered with our optimization task arefl 

flfl An abundance of local minima very close to the global minimumflThe simfl 
plex can get stuck in very smallfhcale holesfleven with starting conditions 
exceptionally close to a known solutionflWe need to allow the optimizer to 
move from a position that is very close to the global minimum towards the 
global minimumfl It seems that this last stage is the difl cult part for the 
NealderfMead simplex methodfllt is possible to produce good scores with 
a number of distinct end points in polynomial parameter spacefl suggesting 
that the problem is probably overffepecifledfl 
flfl This is the polynomial approximation problemflThere will be one polynomial 
that is the best for the stripe straightening but there are a number of distinct 
ones that can approximate it very wellfl 

As to the first itemflthe successful approach to a solution of this problem 
is to employ hybrid techniquesfl Genetic algorithms alone are usually slow in 
optimization problemsfl since they are too coarsegrained to obtain a solution 
quicklyflOn the other handfl downhill algorithms are usually fast fin terms of 
processor cyclesfilif they are close to the solutionflbut tend to get stuck in local 
minimaflCombining both kind of algorithms manages to avoid local minimafland 
finds solutions accuratelyfl 
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Conclusions 

In the task of optimization of parameters of elastic transformation a simplexfl 
method is fastestflbut also the least preciseflgiving the greatest divergenceflThe 
GA technique is the most preciseflbut also requires at least flfl times more timefl 
The hybrid technique is twice faster than GAs at the same accuracyfl 

The intrinsic curvilinear coordinates of an embryo found by our procedures of 
optimization appears to be approximated by contour lines of a map of a gradient 
of the morphogen bicoidfilt is in the good agreement with known ideas about 
a governing role of this gradient in consequent processes of segmentation of an 
early fly embryofl 
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Abstract. This paper presents a genetic algorithm specially designed 
for job shop problems. The algorithm has a simple coding scheme and 
new crossover and mutation operators. A simple local search scheme 
is incorporated in the algorithm leading to a combined genetic algo- 
rithm(CGA). It is evaluated in three famous Muth and Thompson prob- 
lems (i.e. MT6x6, MTlOxlO, MT20x5). The simulation study shows 
that this algorithm possesses high efficiency and is able to find out the 
optimal solutions for the job shop problems. 



Keywords: Combined genetic algorithm, local search, job shop scheduling. 

1 Introduction 

The machine scheduling problem can be defined as a triple{At, ^7, C}, where M 
represents the machine environment, J contains the jobs involved and C is an 
optimality criterion chosen according to the objective of scheduling. A classic job 
shop problem (JSP) [1] [2] consists of M machines on which N jobs are processed. 
Each machine can process only one job at a time, the processing of a job on a 
machine is called an operation. Each job has a specified processing order through 
all the machines with the corresponding processing time on each machine. This 
order is called machine sequence. The scheduling problem is to find out the best 
operation sequences on all machines in order to minimize the makespan. In this 
case, the makespan implies the criterion to be optimized. It is well known that 
the job shop scheduling problem is one of the most difficult NP problem. The 
classic approaches to JSPs are the branch and bound methods(BAB) [3] [4]. The 
performance of the existing algorithms for JSPs is far from satisfactory. 

Over the last two decades, genetic algorithms (GAs) [5] have become more 
and more popular in the field of optimization. The GAs are a class of stochastic 
search algorithms that start with a population of randomly generated chromo- 
somes and evolve towards better new chromosomes by applying genetic operators 
(crossover, mutation and inversion, etc). Two most distinguished characteristics 
of GAs are: (1) Only the fitness of chromosomes need to be calculated. There- 
fore the GAs are suitable for such optimization problems as JSPs which have 
no derivative information provided; (2) The GAs search from a population of 
solutions rather than a single point. They can find out the global optimal solu- 
tion of optimization problems with multimodality efficiently. Development of an 
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efficient coding sheme and design of appropriate GA operators have received a 
great attention in the GA applications [6] ~ [9]. 

Recently, the GAs have been applied to solve JSPs [10] ~ [12]. Local search 
has been tried to work alongside with GA for JSPs [13], but the algorithm is 
rather complicated. 

This paper presents a GA with a simple coding scheme and new crossover 
and mutation operators which are introduced to solve JSPs. A simple local search 
scheme is incorporated in the GA to improve the computation efficiency. In the 
CGA, the offsprings obtained by crossover and mutation are not used in the 
next generation directly but used as a seed of subsequent local search. The 
local search finds out the local optimal chromosomes in the neighborhood of the 
offsprings, then these chromosomes are used in the next generation. The paper 
is organized as follows. Section 2 describes the coding scheme. Construction of 
the new crossover and mutation operators is discussed in section 3. Section 4 
introduces the CGA with local search. Section 6 shows the simulation results. 
Section 5 concludes the paper. 

2 The coding scheme 

2.1 Representation 

In a classic job shop problem, there are N jobs to be processed on M machines. 
Each job must be processed by every machine. Thus there are a total of NxM 
operations. In the coding scheme, each chromosome is a string of NxM genes. 
Each gene represents an operation: 



[genei . . . genet ■ ■ ■ geneNxAi] 



In all chromosomes, the machine sequence for every job is strictly held. This 
constraint is ensured by the population initialization and GA operators. 



2.2 Initialization 

The operations in a chromosome should not be ordered arbitrarily. Eor every 
job, there is a constraint applied from the machine sequence. Eor the scheduling 
problem, we have a machine sequence matrix shown in Eigure 1. The row of 
the matrix represents the machine sequence of the job: 

St — [niji nit2 • • • nitM ] 

In order to make a feasible schedule, the constraints for all jobs must be 
satisfied. The initialization of chromosomes is to take the elements of the machine 
sequence matrix one at a time in the following way: 

(1) Select a non-empty row St (i G N) randomly, mti ^ (j), <j) denotes an empty 
element. 

(2) Let genej = imn,mtk = mt(k+i){k = 1, ...,M - 1),to*m = <t>,3 = 3 + 1- 

(3) Repeat (1) and (2) until nitj = G N,j G M. 
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■ mil 


mi2 


miM 




m2i 


m22 


m,2M 




-ruNi 


niN2 . 


niNM 




1. The machine sequence matrix 



2.3 Evaluation 

As stated above, a chromosome is a string of NxM operations. The job sequence 
of a machine can be obtained easily by scanning a chromosome from left to right 
to find out the job order for the same machine. Repeating the scanning for every 
machine, we have a job sequence matrix, given in Figure 2, which corresponds 
to a chromosome. The row of the matrix represents the job sequence of the 
jnachine: 



Pi = [jii ji2 ••• jm] 

From the machine sequence matrix and the job sequence matrix, an active 
schedule can be achieved in the following way: 

(1) Search a non-empty row Pi {i € M), if = i, job jn will be processed 
as soon as possible after the machine completes the former operation. 

(2) Let jij^ — (A: — 1,...,A^ l),jj 7 v — 4^] Let — 

1, ...,M - = 4 >- 

(3) Repeat (1) and (2) until jik = £ M,k £ N; rriij = € N,j € M. 

The fitness of a chromosome is the makespan of its corresponding active schedule. 



■ Jii 


Jl2 


■ ■ JlN 


j21 


J22 


■ ■ ]2N 


-Jmi 


Jm2 ■ 


■ . 3mn 



Fig. 2. The job sequence matrix 



3 Operators 

3.1 Crossover 

The crossover is an important operator in GAs. The performance of GAs depends 
largely on the construction of crossover operator. By using this operator, a pair of 
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chromosomes (parents) produces a pair of new chromosomes (offsprings) through 
exchanging and recombining schematic information. For JSPs, the difficulty of 
constructing the crossover operator is that the machine sequence of every job 
must be strictly held in newly generated chromosomes. 

The construction of crossover operator for JSPs is as follows: At first, two 
parents are selected; then, k{l<k<N — 1) jobs are selected randomly. These k 
selected jobs correspond to kxM operations in each parent. A string is pro- 
duced from the first parent by picking out these operations with their positions 
unchanged while filling other positions with holes (Hs). Rotating the string for- 
ward or backward I positions. The I positions are determined randomly, but in 
doing so, the relative order of the selected operations and the distance between 
every two selected operations should be maintained. Filling the holes (Hs) with 
the unselected operations of the second parent one by one while the relative 
order of these unselected operations is maintained. Now one offspring has been 
produced. Similarly, the other offspring can be produced with the selected oper- 
ations of the second parent and the unselected operations of the first parent. 

For example, we have the following machine sequence matrix: 

'1 2 3 4' 

3 14 2 

13 2 4 
_3 2 4 1_ 

The following two chromosomes are randomly selected to crossover: 



parentl : 2 ^ 2 i 3 i 43 3 a li 42 32 I 2 I 3 24 22 I 4 44 34 4 i 



parent 2 : li I 2 43 3 i I 3 33 23 42 44 2 i I 4 24 22 32 4 i 34 

Now, select jobs randomly, for example jobl and jobS are selected as follows 
From parentl we have 

string! : H H 3i H 3a h H ^2 l 2 h H H U H 3^ H 
Unselected operations of parentl are as follows 



23 2i 43 42 24 22 44 4 i 



From parent2, we have 

string2 : h U H 3i I 333 H H H H U H H 32 H 3^ 
Unselected operations of parent2 are as follows 



43 23 42 44 2 i 24 22 4 i 

Then rotate string! randomly. For example, one position forward, leave the last 
position as a hole, then we have 

string3 : H 3i H 3a h H 32 UI 3 H H U H 3^ H H 

Rotate string2 randomly. In this case, zero position is the only choice , then we 
have 

stringA : Ij 12^1 3i I 333 H H H H UH H 3a H 3i 
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Fill in the holes of string^ with the unselected operations of parents, we have 
one offspring 

offsprl : 4 s 3 i 2s 83 li 42 82 I2 I3 44 2i I4 24 84 22 4 i 

Fill in the holes of string^ with the unselected operations of parentl, we have 
another offspring 

ojfspr2 : li I2 23 3 i I3 83 2i 43 42 24 I4 22 44 82 4 i 84 



3.2 Mutation 

Mutation is a random walk through the solution space, although it plays a sec- 
ondary role in GAs, it is an insurance policy against premature loss of important 
information, a well constructed mutation operation can help a GA to get a better 
solution more easily and more fast. 

Relocation is the key of the mutation operator. First, M operations with the 
same job are selected randomly; then, their positions are all relocated randomly 
at the same time in such a way that the order of these selected operations and 
the distance between every two selected operations are strictly held. The order 
of those unselected operations is also held. 

For example, the following chromosome is to be mutated 



oldstri : 23 2i 3 i 43 83 li 42 82 I2 I3 24 22 I4 44 84 4 i 



Operations with job 1 are selected, then we have the following string 
string 5 ■. H H H H H U H H h I3 H H U H H H 
Unselected operations are as follows: 

23 2i 3 i 43 83 42 82 24 22 44 84 4 i 

Now, relocate the first selected operation(li) randomly, for example, at position 
3 (in this case, its new position can be chosen from 1 to 9 ), and relocate other 
selected operations accordingly, we have 

stringQ ■. H H U H H hh H H UH H H H H H 

Fill in the holes with the unselected operations in their previous order, the new 
chromosome is now produced 



newstri : 2^ 2i li 3 i 43 I2 I3 83 42 I4 82 24 22 44 84 4 i 



3.3 Reproduction 

Reproduction is basically a process in which individual chromosomes are copied 
according to their fitness values. The classic reproduction method is the biased 
roulette wheel method. This method is simple and does well in many practical 
problems, but the simulation study shows that it is not efficient for job shop 
scheduling problems, because it forms a very large searching space. The repro- 
duction applied in the proposed GA introduces new chromosomes and reinforces 
the principle of best survival, which are addressed as follows: 
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(1) New chromosomes 

Some chromosomes are selected randomly and replaced by new randomly 
generated chromosomes. By doing so, new information will be brought in to 
prevent premature. Simulation shows that this is very useful in solving job 
shop scheduling problems. From the simulation study, it is found that two 
or three new chromosomes used in every generation would provide better 
performance. 

(2) Best survival 

— The best chromosome of a generation is kept to evolve into the next 
generation. 

— In crossover, two best chromosomes are selected from the two parents 
and the two offsprings to survive into the next generation. 

— In mutation, the best from the old chromosome and the new chromosome 
is selected to survive into the next generation. 

4 Genetic algorithm with local search for JSPs 

4.1 Local search 

Local search(LS) is a method to find the best solution from a neighborhood 
region. A solution x represents a point in the search space, and its neighborhood 
N{x) is defined as a set of feasible solutions capable of being reached from x by 
exactly one step transition. For the min /(a;) problem, the local search method 
is described as follows: 

One-step local search(OSLS) 

(1) Select point x. 

(2) Determine the neighborhood N{x) of x and evaluate f{x), x £ N{x). 

(3) Find x* from {x,N{x)} according to: f{x*) = min f{z). 

z^{x,N {x)} 

Multi-step local search(MSLS) 

(1) Select a starting point Xi,i = 0. 

I2l Determine the neighborhood N{xi) of Xi and evaluate f{xi), Xi £ N{xi). 

(3) Find x* from {a;,, A" (a;,)} according to: f{x*) = min f{z)- 

ze{xi,N(xi)} 

(4) Let i = i + l,Xi = X* , repeat (2) to (4) until the termination condition is 
met. 

Figure 3 illustrates the multi-step local search method. 

4.2 GA/LS for JSPs 

In our algorithms, the neighborhood of a chromosome is defined as the set of 
all feasible chromosomes produced by changing the positions of two neighboring 
genes. So the determination of the neighborhood of a chromosome is rather easy. 
The local search is incorporated in the GA. The GA/LS for JSPs is described 
as follows: 

(1) Initialize the population. 

I2l Make crossover on selected chromosomes, do local search on the offsprings. 

(3) Make mutation on selected chromosomes, do local search on the offsprings. 

(4) Reproduce a new population. 

(5) Repeat (2) to (4) until the termination condition is met. 
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Fig. 3. Multi-step local search 
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Fig. 4. The best convergence curves for MTlOxlO 



Table 1. Minimum makespans for job shop problems 



Algorithm 


MT6x6 


MTlOxlO 


MT20x5 


Barkerl985(BAB) 


55 


960 


1303 


Carlierl989(BAB) 


55 


930 


1165 


Nakanol991(GA) 


55 


965 


1215 


Federicol995(GA) 


55 


946 


1178 


This paper(CGA) 


55 


930 


1165 
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5 Conclusion 

6 Simulation results 

The above genetic algorithm has been implemented in C++ to solve three well- 
known JSP benchmarks [14]: 

— MT6x 6(6 jobs, 6 machines) 

— MTlOx 10(10 jobs, 10 machines) 

— MT20x5(20 jobs, 5 machines) 

The minimum makespans achieved by this algorithm for the three benchmarks 
(MT6x6, MTlOx 10, MT20x5) are 55, 930 and 1165 respectively. For every 
benchmark, more than one optimal schedules with the same minimum makespan 
have been found out. Figure 4 shows the convergence of the CGA, in which the 
size of the population was chosen as 168, and the crossover and mutation prob- 
abilities were given as 0.7 and 0.2 respectively. It can be seen from Figure 4 that 
the CGA provides a much better convergence performance and the OSLS and 
MSLS offer different contributions to the CGA. Table 1 compares the minimum 
makespans obtained by this algorithm and other existing algorithms. The result 
of CGA is the same as Carlier’s BAB and better than other algorithms. But 
it is well known that the large amount of computation of BAB for JSPs is a 
notorious problem. Three optimal schedules for the three JSP benchmarks are 
presented in Table 2, Table 3 and Table 4 respectively. In the tables, ‘start’ and 
‘end’ represent the starting time and the end time of a job on a machine. 

A combined genetic algorithm(CGA) with a simple coding scheme and 
specially designed crossover and mutation operators has been proposed to solve 
job shop scheduling problems. A simple local search scheme is incorporated in 
the algorithm. The optimal schedules for all the three JSP benchmarks [14] can 
be achieved by the proposed CGA. The simulation results show that the new 
crossover and mutation operators are efficient. The reproduction involving the 
‘new chromosomes’ has been proved to be useful. It speeds up the searching 
process and guarantees the optimal results to be found. The algorithm obtains a 
better convergence by the introduction of the incorporated local search scheme. 
The shortcoming of the simple coding scheme is that there are more than one 
chromosome applied for a same schedule. This actually results in an increase of 
the searching space. 
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Abstract. This paper presents a distributed learning control strategy 
for traffic signals. The strategy uses a fully distributed architecture in 
which there is effectively only one (low) level of control. Such strategy 
is aimed at incorporating computational intelligence techniques into the 
control system to increase the response time of the controller. The idea is 
implemented by employing learning classifier systems and TCP/IP based 
communication server, which supports the communication service in the 
control system. Simulation results in a simplified traffic network show 
that the control strategy can determine useful control rules within the 
dynamic traffic environment, and thus improve the traffic conditions. 

1 Introduction 

Traffic control in large cities is a difficult and non-trivial optimization prob- 
lem. Most of the existing automated urban traffic control systems, such as 
TRANSYT[1], SCATS[2], LVA[3] and SCOOT[4], have a centralized structure, 
i.e. information gathering and processing, as well as control computations, are 
carried out in a centralized manner, in which case efficiency is decreased due to 
the large volume and the heterogeneous character of information [5] . To achieve 
global optimality, hierarchical control algorithms are generally employed. How- 
ever, these algorithms have a slow speed of reaction and it has been recognized 
that incorporating some computational intelligence into lower levels can remove 
some burdens of algorithm calculation and decision making from higher levels 
[6]. Recently, there is a growing body of work concerned with the use of evo- 
lutionary computing techniques for the control of traffic signals. Montana and 
Czerwinski [7] proposed a mechanism to control the whole network of junctions 
using genetic programming [8]. They evolved mobile “creatures” represented as 
rooted trees which return true or false, based on whether or not the creature 
wished to alter the traffic signal it has just examined. Cao et al has developed an 
intelligent local traffic junction controller using learning classifier systems and 
fuzzy logic [9] and showed that the local controller can determine useful junc- 
tion control rules within the dynamic environment. Mikami and Kakazu used 
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a combination of local learning by a stochastic reinforcement learning method 
with a global search via a genetic algorithm. The reinforcement learning was 
intended to optimize the traffic flow around each crossroad, while the genetic 
algorithm was intended to introduce a global optimization criterion to each of 
the local learning processes [12]. Escazut and Fogarty proposed an approach to 
generate a rule for each junction using classifier systems in biologically inspired 
configurations [13]. 

This paper is devoted to developing a distributed learning control strategy 
for traffic signals. A fully distributed architecture has been developed in which 
each subsystem is solely responsible for one aspect of the system and where a 
coherent global control plan emerges from the interactions of the subsystems; 
no hierarchical structure is included. Such an approach is aimed at increasing 
the speed of response of the local controller to changes in the environment. To 
do this, we have developed an agent-alike controller, which is implemented by 
employing learning classifier systems [14, 15] and TCP/IP based communication 
server, supporting the communication in the control system. Simulation results 
in a simplified traffic network show that the control strategy can determine useful 
control rules within the dynamic traffic environment, and therefore improve the 
traffic conditions. 



2 An Agent-alike Controller 

Optimization of a group of traffic signals over an area is a large and multi- 
agent type real-time planning problem without precise reference model given. 
To do this planning, each signal should learn not only to acquire its control 
plans individually through reinforcement learning but also to cooperate with 
each other. This requires communication between the agents. If each signal 
simultaneously communicates with each other and controls its phases according 
to the change of the global traffic flow, the total volume of the area will be 
well optimised. However, to provide the efficient communication is a difficult 
task, caused by the inefficient accounting of interactions between subsystems (in 
decentralized case) and the complex communication structure (in the hierarchical 
case). 

In this work, we developed an agent-alike controller, consisting of a learning 
classifier system and a communication server, as shown in Figure 1. Rule-based 
controller as classifier systems lie midway between neural network and symbolic 
processing systems that can combine the benefits of both. The limitation of 
specifying a single classifier system is that while it may work for a simple con- 
troller, the method does not scale up to complex control systems. Work done so 
far has addressed this problem by dividing a complex system into its simplest, 
physical sub-systems, specifying rule-based controller for each of these and thus 
creating a multi-agent system. 

For traffic control problem, we associate an agent to each junction of the 
traffic network. So, the whole control strategy developed in this work contains a 
number of distributed, communicating agents, where each agent has a classifier 
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Figure 1: Structure of the agent-alike controller 



system (providing the control strategy) and a communication server which is 
used to connect the agent to the user interface, the application and to other 
agents. 

The two elements of the agent, classifier system and communication server, 
are separate since; as messages are passed around the agent network, the commu- 
nication server acts independently of the classifier system to route the message 
to its neighbours. Another reason for keeping the communication server distinct 
from the classifier system is that communication is likely to be implementation 
specific (even in the test applications) . Thus it is necessary when specifying the 
communication server to consider the general requirements of setting up and 
maintaining the communication in a distributed learning system rather than 
those in a specific software and hardware implementation. 

2.1 Classifier systems 

A classifier system is a learning system in which a set (population) of condition- 
action rules called classifiers compete to control the system and gain credit 
based on the system’s receipt of reinforcement from the environment. A classi- 
fier’s cumulative credit, termed strength, determines its influence in the control 
competition and in an evolutionary process using a genetic algorithm in which 
new, plausibly better, classifiers are generated from strong existing ones, and 
weak classifiers are discarded. 

A classifier c is a condition-action pair 

c =< condition >:< action >, 

with the interpretation of the following decision rule: if a current observed state 
matches the condition, then execute the action. The condition is a string of 
characters from the ternary alphabet {0, 1, # }, where # acts as a wildcard 
allowing generalization. The action is represented by a binary string and both 
conditions and actions are initialized randomly. The real-valued strength of a 
classifier is estimated in terms of rewards obtained according to a payoff function. 
Action selection is implemented by a competition mechanism, where a strength 
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Figure 2: Initialisation structure of communication server 



proportionate selection method is usually used. To modify classifier strengths, 
the given credit assignment algorithm is used, e.g. the Bucket brigade [14]. 

To create new classifiers a standard GA is applied, with three basic genetic 
operators: selection, crossover and mutation. The GA is invoked periodically 
and each time it replaces low strength classifiers with the offspring of the selected 
fitter ones (the reader is referred to [14] for full details). 

2.2 Communication Server 

The communication server provides the service for each agent connecting to the 
user interface, application and other agents. All these channels might involve 
two-way communication. During the initialisation the agents open a communi- 
cation channel and await a connection message. The channel is tested to ensure 
the communication is setup correctly as although the configuration parameters 
have been previously checked for consistency, the parameters may be inconsistent 
with the physical communication process, also this process might be faulty. 

Although the term socket is used in the specification as the medium to 
connect communication channels in implementation other methods can be used, 
such as calls to remote objects, when using RMI or DCOM. The basis of the 
communication initialisation and run-time processes are not affected. The cre- 
ation of the communication object and binding in a remote registry (on a given 
hostname and port) replaces the creation of a server socket and calls to the 
remote object replace read and write calls to the sockets. 

Note that it is possible for the communication server to create separate 
processes to listen on the communication channel for messages, this allows the 
agent to be reactive to external messages. 

The communication with neighbours requires a single channel for incoming 
messages, and separate channels from sending to each neighbor (except if the 
messages are broadcast on sent via a proxy) . The initialisation of the communi- 
cation server, shown in Figure 2, involves the following steps: 
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1. The Communication Server object creates the specific Communication Services 
(Application, User Interface or Neighbourhood) as specified by the configuration. 

2. The Communication Server object create a monitor which maintains the list of 
current connections 

3. The Communication Server object passes the Communication Service object 
and connection configuration information to the Connector object which, for 
connection with the User Interface and Application and incoming channel from 
the neighbouring agents, opens a Server Socket on the specified port and waits 
for a request to connect. For the outgoing channel to the neighbouring agents the 
Connector object intermittently requests a connection to the neighbours specified 
port.. 

4. The User Interface, Application or Neighbouring Agent sends a request to con- 
nect. 

5. The request to connect is accepted by the neighbour’s server socket. 

6. The Connector sends the Communication Service object and open socket to the 
Communication Monitor. 

7. The Communication Monitor object tests the communication channel, if the 
test succeeds the Communication Service is passed to the Connection object, 
otherwise the socket is closed and the failure reported. 

8. The Connection object starts the thread to handle the connection and passes 
the input and output streams to the Communication Service object. 

3 How to Control Traffic Signals 

To control a traffic network, we associate an agent to each junction of the traffic 
network. The agents are initialised according to the traffic network configuration 
and user-specified parameters. For the simulated 2x2 traffic network, shown in 
Figure 3, four agents, i.e., agents I, II, III, and IV, associating with junctions I, 
II, III, and IV, are need to provide comprehensive control of the network. Agent I 
has the neighbouring agents II and III, and agent II has the neighbouring agents 
I and IV, etc. The communication server in each agent provides the control 
actions of its neighbouring agents, and these information is used to construct 
control rules for its junction. 

The classifier system employed is a version of Wilson’s “zeroth-level” system 
(ZCS) [15]. ZCS is a Michigan-style classifier system, without internal memory. 
In order to avoid the genetic algorithm manipulating unnecessarily long rules, we 
extend the binary string representation in ZCS to a more general representation, 
which uses 0 to L (L < 10) for each variable (bit) position instead of the binary 
code. This reduces the string length significantly and appears to benefit multiple 
variable problems. For these hybrid strings, mutation in the GA is performed 
by changing an allele to a randomly determined number between 0 and L other 
than itself [16]. 
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Figure 3: The simulated traffic environment 

3.1 Individuals 

The classifiers have the representation shown in Figure 4. The condition part of 
each classifier consists of six bits, which reflects the scalar level of queue length 
from each direction and the previous actions of the neighbouring agents. In this 
application, the scalar level of the queue length is set to 4, which ranges from 0 to 
3, corresponding to the four linguistic variables, {zero, small, medium, large }. 
The action part indicates the required state of the signal. For instance, for 
junction I, the rule 130201:1 says that if the queue from directions east and west 
are small (1) and zero (0), but the queue from directions south and north are 
large (3) and medium (2), and the previous neighbourhood junction controllers’ 
actions are vertically red (0) (junction II) and green (1) (junction III), then the 
traffic light stays green vertically (1) for a fixed period of time. 

3.2 Evaluation of actions 

We assume that the junction controller can observe the performance around it, 
let the evaluated performance be P. Traffic volume sensors are set at each of 
the intersections. They are able to count the numbers of the cars that come 
from all directions, pass through the intersection and stop at the intersection. 
In this study, the evaluation function we use to reward the individuals is the 
average queue at the specific junction. Let qi denote the queue length from 
direction i at the intersection {i = 1, 2, 3, 4), then the evaluation function is: 
/ = { Qi- We thus attempted to minimize this measure. Let us identify 
the A:-th cycle by a subscript k, then fk for the cycle k is calculated by observing 
the sensor from the beginning of the A:-th cycle to the end of this cycle. Thus, 
the evaluated performance of the action performed at the A:-th cycle is computed 
as Pk = fk-i — fk- Specifically, if Pk > 0, the matched classifiers containing the 
performed action should be rewarded, otherwise penalized. 
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Figure 4: Structure of the classifier system rules 

3.3 Reinforcement learning 

After the controller has produced an action, the environment judges the output, 
and accordingly, gives payoff in the following manner: 

• Rewards: The objective of the local signal controller is to minimize the 
average queue length, fi. We have found the performance-based rewards 
are helpful in the environments we used in our experiments. The reward 

function we used was r = \ ((100 “ Qk,i) + (Qk,i ~ qk-i,iY ^ , where 

Qk,i denotes the queue length of the Rh direction at the kth cycle. 

• Punishments: We use punishments (i.e., negative rewards). We found the 
the use of appropriate punishments results in improved performance (in a 
fixed number of cycles), at least in the environments used in our experi- 
ments. We also found that large punishments could lead to instability of 
the classifiers and slow convergences of the rules. The appropriate punish- 
ments should be determined by trial tests. 



4 Simulation Results 

For the traffic network shown in Figure 3, we developed a simplified traffic sim- 
ulator, which is similar to the one used in [12]. The simulator is composed of 
four four-armed junctions and squared roads. Each end of a road is assumed to 
be connected to external traffic, and cars are assumed to arrive at those roads 
according to a Poisson distribution. Each intersection has two “complementary” 
signals: when the horizontal signal is red, the vertical signal is green and vice 
versa. Each of the cars attempts to attain the same maximum speed. When a 
car passes an intersection, it changes its direction according to the probability 
associated with that intersection. Specifically, let di, i = 1,2,3, be the next 
directions for a car, that is, {di} = { right, forward, left }. At each of the inter- 
sections, the probabilities {pd ^ } are previously given, where pd^ corresponds to 





124 



Y.J. Cao et al. 




(a) Number of Cars = 30 




(b) Number of Cars = 60 



the probability of selecting an action di for the car passing through the intersec- 
tion. Roads are not endless, thus only a limited number of cars is allowed to be 
on the road at a given time. If a car reaches the end of the road, then the car is 
simply removed from the simulation, and another car is generated, entering on 
a randomly selected road. 

For comparison purpose, two types of control strategies are employed: ran- 
dom control strategy and the developed distributed learning control (DLC) strat- 
egy. The random control strategy determines the traffic light’s state ( 0 or 1) 
randomly at 50% of probability; whilst distributed learning control (DLC) strat- 
egy determines the traffic light’s state according to the action of the winning 
classifier of the agent. The parameters used for the DLC were as follows: 

• Population size: 100 

• Mutation probability: 0.05 

• Crossover probability: 0.85 

• Selection method: Roulette wheel selection 
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(c) Number of Cars = 90 

Figure 5: Comparison performance of the control strategies 



As the major task is to test whether the proposed DLC can learn some good 
rules in the traffic network, experiments were carried out for three different types 
of traffic conditions. In these simulations, the mean arrival rates for the cars 
are the same but the number of cars in the area is limited to 30, 60, and 90, 
corresponding to a sparse, medium, and crowded traffic condition. In all cases, 
the DLC strategy is found to learn how to reduce the average queue length 
and improve the traffic speed in the network. For example. Figure 5 shows 
the average performances of the random control strategy and DLC strategy 
respectively over 10 runs in all cases, where the solid line represents DLC strategy 
and the dotted line represents random control strategy. It can be seen that the 
DLC strategy consistently learns and improves the traffic speed over 10,000 
iterations. 



5 Conclusion and Future Work 

In this paper we have presented a distributed learning control strategy for traffic 
signals. The simulation results on a simplified traffic environment are encourag- 
ing since we have shown that the developed control strategy can learn to coor- 
dinate and determine useful control rules within a dynamic environment. This 
preliminary work needs, of course, a number of extensions. We are currently 
extending this work in a number of directions, particularly examining ways of 
improving the learning capability of classifier systems and the performances in 
much more complicated traffic network. 
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Abstract. Time-series prediction and forecasting is much used in engineering, 
science and economics. Neural networks are often used for this type of 
problems. However, the design of these networks requires much experience and 
understanding to obtain useful results. In this paper, an evolutionary computing 
based innovative technique to grow network architecture is developed to 
simplify the task of time-series prediction. An efficient training algorithm for 
this network is also given to take advantage of the network design. This 
network is not restricted to time-series prediction and can also be used for 
modelling dynamic systems. 



1. Introduction 

Dynamic modelling addresses the modelling problem from data of a dynamic system. 
A dynamic system is a system which has internal states represented in an abstract 
phase or time space. Its future state and outputs depends on its current state. They 
can be mathematically described by an initial value problem [1]. An example of 
dynamic modelling is the modelling of time series data, where predictions has to be 
made on the future values of the time series based on current values of the series. 
This type of modelling tries to capture the geometry and geometrical invariants of a 
dynamic system from past outputs of the system [2]. The use of past outputs, delay 
co-ordinates, to model dynamic systems can be traced back as far as 1927 to the work 
of Yule, who used auto-regression (AR) to create a predictive model for sunspot 
cycles [1], [3]. 

The most popular method of modelling time-series data today is the statistical 
method of Box-Jenkins. The Box-Jenkins methodology search for an adequate model 
from AR, moving average (MA), auto-regression moving average (ARMA), and auto- 
regression integrated moving average (ARIMA) [5]. This modelling is a three-stage 
process: identification, estimation, and diagnostics. 

The identification involves the use of sample auto correlation functions (SACF) 
and sample partial auto correlation functions (SPACE) to analysis the linear 
relationships of the time series with its lagged images. The estimation process 
involves finding a model, one of AR, MA, ARMA, and ARIMA, with a theoretical 
ACF and PACF similar to the SACF and SPACE of the time series. The third stage, 
diagnostics, involves residual analysis, goodness of fit statistics, and cross-validation. 
The limitation of this method is that human decision and associated errors are inherent 
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in every stage of the process [6]. Also the SACF and SPACF functions measure only 
linear relationships. 

Neural networks have been shown to be universal function approximators [4]. By 
using delay co-ordinates as the inputs of a neural network, it can be used as a non- 
linear approximator for a delay differential equation. With an appropriate structure 
and learning strategy, this delay differential equation can be tuned to have similar 
behaviour to the dynamic system being modelled. 

Neural networks are commonly used for time-series predictions also. Different 
network architectures have been employed to tackle the prediction of time-series data, 
for example the multi-layer perceptrons [7], the finite-duration impulse response 
(FIR) networks [8], and the recurrent networks [9]. Good prediction results have been 
obtained from such networks. The difficulty with these architectures is that they are 
static, fixed before the training is begun. That is, the designer needs to decide on the 
number of delay co-ordinates to use, the number of hidden neurons to have, etc. 

In this paper, a novel network architecture and training strategy is proposed for the 
modelling of dynamic systems which alleviates the designer of much of these 
decisions. The network structure is not static and changes during the training process 
which makes use of evolutionary algorithms (EA). A description of the novel 
network architecture is given in Section 2. The evolutionary technique for growing 
and training the network is described in Section 3. In Section 4, results of predicting 
chaotic time series using this technique and network are shown. Section 5 
demonstrates the application of time series prediction on real data, and shows a 
method of improving prediction. Finally, the paper is concluded in Section 6. 



2. Lateral Delay Neural Network 

A neural network requires memory to have dynamic behaviour [8]. This memory can 
be delay elements in the architecture or the use delay co-ordinate inputs. The number 
of delay co-ordinate inputs relates to the embedding dimension of the system, and the 
number of hidden neurons is dictated by necessary degrees of freedom [10]. Both of 
these have to be estimated for static network architectures. 

The principle of growing the network structure one hidden neuron at a time has 
shown to be a fast and efficient method of approximating a function with a neural 
network [11]. Several such incremental learning algorithms have been implemented 
[11], [12]. It is desirable to use these incremental approaches to speed the learning 
process and reduce the dimensionality of optimisation. Yet there are two structural 
parameters that needs to be estimated, namely, numbers of delays and hidden neurons. 
The design of the lateral delay neural network (LDNN) combines these two 
parameters into one for the implementation of incremental learning algorithms. This 
allows for the increased speed and efficiency of modelling of dynamic systems. 

The combination of delay elements and hidden neurons is shown in Fig. 1 of the 
network architecture. This architecture is incremental with the hidden neurons 
forming a one-way chain of delayed elements. The simplicity of design is facilitated 
by not requiring structural decisions. Improving the performance of network is 
achieved by adding a neuron to the lateral chain. There are very few synaptic weights 
associated with each neuron, thus the complexity of the weight optimisation is very 
low. 
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Fig. 1. Architecture of LDNN. 

The LDNN can be used in dynamic modelling with its inputs jcg SR™ being the m 
system inputs and its output y being the predicted system output. In time-series 
prediction, there is only one input which this the current value of the time-series, and 
the output is the predicted future value of time-series, i.e., a: = [x„] and = y , 
where is the rt data in the series. 



3. Evolutionary Network Growing and Training 



3.1. EA-only Training 

Due to the LDNN architecture’s simplicity and incremental design, training 
algorithms can be devised to take advantage of these properties. EAs can be used to 
train the network incrementally by first optimising the synaptic weights of the 
network with only one hidden neuron, i.e., no delay elements yet. The best solution 
found is then used to “hot-start” subsequent EA optimisations of the network with an 
added hidden neuron. This can be iterated until the desired accuracy is reached or 
until over-fitting begins to occur. 

Using EA for training in this way can produce good results, but the dimensionality 
of the optimisation space increases rapidly. If there is a weight associated with each 
synaptic connection and one for the threshold of each neuron, then the dimensionality 
involved is (mH-3)n-l where m is the number of inputs and n is the number of hidden 
neurons. 
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3.2. Orthogonal Training with EA 

The dimensionality of the optimisation can be greatly reduced by keeping the 
optimised weights of the previous network fixed and only optimising the weights of 
the newly added neuron. A learning algorithm based on this idea is given for this 
LDNN here. 

Beliczynski gave an incremental algorithm for training one-hidden-layer 
perceptrons [11]. This algorithm is slightly modified to accommodate the learning of 
LDNNs and lateral delays; the proofs can be found in his work [11]. 

Thus, first assume that the time series to be modelled is defined by a finite set of 
input-output pairs: 

( (Xj, /(flj, Xj)), (Xj, /(a^, X,)), (x„, /(a„, xj) } (1) 

where/: 91 ^ 91 is the system output (the next data), x is an input (the current 

data), X G 91 , a is the internal state of the system, a g 91 \j = 1, N, and Ais the 
number of input-output pairs. It follows that for time series, 

( 2 ) 

where j = 1, ..., N-l. Define the vector X of inputs, i.e., the time series data in 
order, as 



A= [x„X2, ...,xJ'’g 91" 


(3) 


and output vector 




f(A) = [/(«!, Xi),/(a 2 ,X 2 ),...,/(a„,x„)f g 91" 

= [x2,X3,...,X^^lf 


(4) 


Let g/ j. , x^) denote the function of a hidden neuron, where : 91 ' ' ^ 91 , 
is the delayed input to the hidden neuron, for every neuron k= 1, n, and time j = 1, 

N, where n < and n is the maximum number of hidden neurons allowed. 

According to the structure of the network 


„ fO if y = l, or/c = l 

i k ~ \ 

otherwise 


(5) 


where k = 1, ..., n, and j = 1, N. The delays of the network constitutes the 

internal state of the network, thereby the network state vector dj and the state matrix 


can be defined as follows 




dJ = [dJ^,dJ2,..■,dJ„f eW 


(6) 


A„ =[ai,a2,-.-,«A,f e 91"'" 


(7) 



Define the vector G/A) which is composed of the outputs of the hidden neuron 
for the whole time series 
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G,{X) = [g, (^1 , , Xi ), g, (fl2,r {a,,, , )]^ G SR ~ 

/V /V yv yv J’ 

“ [®2,*+l ’ ^3,k+l > • • • > ‘^N.k+l > ^ > Xfj )] 

and define the matrix 

H„(X) = [G,(X), G,(X),..., G„(X)] G ( 9 ) 

Also let denote the vector of weights of the output neuron 

W, = [h’„W2, ...,wJ"g SR" (10) 

where is the weight of the synaptic connection from neuron k to the output 
neuron, k = \,...,n . 

Thus the network’s predictions, FJ^X), of the time series X is 

F (Z)=H (Z)W G SR" (11) 

Now the training error in network prediction can be defined as 

_\F{X) forn = 0 

^"^^^■|f(Z)-F„(Z) forl<n<«„„ 

and the mean squared training error is 

c„(Z) = ^||F„(Z)f (13) 



From [11], |£'^(X)| is non-increasing, and the maximum rate of decrease in 
|,E„(Z)|| occurs when the newly-added neuron, is chosen such that 



sup 

g€G 



EAXf 



G„.,(X) 



G„,i(Z) 



( 14 ) 



is achieved. This is provided that < N. In practice, N has to be large for time- 
series prediction and « N. The quantity in Equation 14, is the scalar product of 
the two vectors, and by maximising it, the two vectors are made quasi-parallel. The 
error is also orthogonal to the output of every hidden neuron in the network. 

There are two optimisations to be performed for each new neuron being added. 
The optimisation of the newly added hidden neuron, g^, and the weights of the 
output neuron. These are optimised using EAs. The dimensionality of these search 
are m+l for g^ and «H-1 for where m is the number of inputs and n is the total 
number of hidden neurons. The whole orthogonal incremental training process is 
summarised in the flowchart in Fig. 2. 

The hidden-layer neuron function, Xj), can use different activation 

functions. Neither of the two training algorithms discussed in this section have many 
restrictions on the activation functions used for the hidden neurons. For all the time- 
series predictions made in this work, sigmoid functions have been used as activation 
functions for the hidden neurons. A linear function is chosen for the output neuron’s 
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activation. The use of sigmoid activation functions means that the network output is 
bounded, thus the time-series being modelled must also be bounded. 

The training algorithms in this section are static; the network does not change after 
the training process. Thus the systems that can be modelled are autonomous systems 
and the time-series that can be predicted are stationary. 




Fig. 2. Flowchart of the orthogonal incremental training process. 

The results shown in Fig. 3 are the times taken to train an additional neuron by the 
two training algorithms as the number of neurons increases. These times are the 
average times of 5 network trainings for the prediction of the Mackay-Glass chaotic 
time-series. The additional training time taken by the orthogonal training algorithm, 
with reduced complexity, stays near constant - rising only slightly as the number of 
hidden neurons increases. Whereas, with the EA-only training, the extra time taken 
per neuron increases exponentially with the addition of hidden neurons. 
Experimentation shows that the EA-only algorithm often produces very good 
prediction accuracy with very few neurons, while the orthogonal algorithm may need 
many more neurons to achieve the same level of accuracy. Due to the low 
computational power required by the orthogonal training algorithm, the rest of the 
prediction results in this paper are obtained using this training method. 
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Fig. 3. Comparison of training times taken by the two training algorithms when adding new 
neurons. 



4. Prediction of Chaotic Series 

4.1. Logistic Map 




Fig. 4. RMS validation errors of LDNN prediction 1 step and 2 steps into the future of the 
logistic map. 

The logistic map is a chaotic series given by the equation 

=4x„(l-x„) (15) 

Using the orthogonal training algorithm, a LDNN is trained to predict the logistic 
map 1 step into the future, and another LDNN to predict 2 steps into the future. The 
resulting validation errors of prediction, of data unseen during training, for up to 10 
hidden neurons are plotted in Fig. 4. The use of incremental techniques guarantees 
that the training error monotonically decreases. The validation errors show the 
model’s generalisation capability. The prediction validation values of the 10 hidden 
neuron network are compared with the actual values in Fig. 5, showing a close match. 
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Fig. 5. 1 step and 2 steps ahead logistic map prediction results with 10 hidden neurons. 



4.2. Mackay-Glass Series 

The Mackay-Glass differential equation produces chaotic time-series that are 
quasi-periodic. It is defined as 

ax,_^ 

^ = ( 16 ) 

The decrease of prediction validation errors with the increase of hidden neurons 
can be seen in Fig. 6 for the prediction of the Mackay-Glass series up to 5 steps into 
the future. The prediction validation results are shown in relation to the actual data 
values for 1, 3, and 5 steps ahead prediction in Fig. 7. It can be seen that prediction 
results are very close to the actual data values, though the predict results get more 
erratic the further into the further the prediction is. 

All these predictions are made with only one input to the LDNN. It may be 
possible to improve on these results by explicitly embedding other distant delay co- 
ordinates as inputs to the network, for example x,_j 5 , etc. 
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Fig. 6. RMS validation errors of predictions of the Mackay-Glass series at several steps into the 
future. 




Fig. 7. 1,3 and 5 steps ahead Mackay-Glass prediction results with 10 hidden neurons. 



5. Prediction of Sunspot Numbers 

Sunspots numbers are indicators for the level of solar activity in the Sun. The sunspot 
numbers form a cycle of approximately 1 1 years. Here, yearly sunspot numbers are 
used to train a LDNN, demonstrating the use of this network and its ease of 
application. The data used are yearly numbers from 1851 to 1998 [13]. 
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Fig. 8 shows the RMS errors of prediction validation as the number of hidden 
neurons increases. Using incremental training guarantees the decrease in RMS 
training errors, and the analysis of validation errors will show the generalisation 
ability of the LDNN model. From Fig. 8, the validation errors appear to start 
continually increasing when the number of hidden neurons reached 6-7. Further 
addition of hidden neurons will most likely decrease the model’s generalisation ability 
and increase over-fitting to the training data. 




Fig. 8. RMS validation errors for the prediction of yearly sunspot numbers. 

To improve on these results, one can choose to use a different activation function 
for subsequently added hidden neurons on the detection of over-fitting. Alternatively, 
explicitly embedding delay co-ordinates as inputs to the network can be employed. 
The latter approach is used in this case and the results of Fig. 8 can be used to suggest 
which delay co-ordinate to embed. Since over-fitting occurs at about the addition of 
the 7* hidden neuron and hence the best network outputs depend on only up to the 7“' 
delay co-ordinate, the 8* delay co-ordinate is chosen, i.e., 

Xn^l= ( 17 ) 



Xn+2= ( 18 ) 

where is the internal state of the model, and x^ is the value of the time series. 




Fig. 9. RMS validation errors with the explicit embedding of an extra delay co-ordinate. 






Time Series Prediction by Growing Lateral Delay Neural Networks 137 



The resulting decrease in RMS validation errors with increasing number of hidden 
neurons can be seen in Fig. 9 when using an extra delay co-ordinate. Over-fitting 
does not occur so soon with this arrangement and generalisation is good. The training 
and validation results for 1 and 2 steps ahead predictions are shown in Fig. 10 and 1 1 
respectively. 




Fig. 10. Training and validation results for 1 step ahead prediction of sunspot numbers. 




Fig. 11. Training and validation results for 2 step ahead prediction of sunspot numbers. 



6. Conclusions 

A novel architecture and growing technique are proposed in this paper which 
simplifies the design process of neural networks for time-series prediction. The 
evolutionary incremental training algorithm presented for this network architecture 
shows itself to be a fast and efficient method of approximating the network to a 
dynamic system. The training algorithm allows for different combinations of 
activations functions to be used in the network. It is possible that a change in hidden 
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neuron activation function will increase the rate of error reduction, when this rate is 
converging. 

In addition to the prediction of time-series, the network may be used for the modelling 
of dynamic systems. Improved results may be obtained with the addition of 
exogenous variables or explicit embedding of delay co-ordinates. With static training 
as given in this paper, autonomous systems and stationary time-series can be 
modelled. This methodology is validated by the application to a real example, namely 
the prediction of sunspot numbers. Its ease and efficiency is demonstrated along with 
explicit embedding of delay co-ordinates to improve results. For the modelling of 
non-autonomous systems and non- stationary time-series, real-time adaptation of this 
network may be required, which is a subject of ongoing research here in CSC, 
University of Glasgow. 
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Abstract. Chssical controllers are highly popular in hdustrial applicat ions. 
However, most controllers are tured manually h a trial and error process 
thoughcomputer smulation. Thiis prticularlydifficult when the sysEmtobe 
controlled b nonlinear. Toaddress ths problem and help (tsign ofindustrial 
controllers fora wider ange of operating tnjectory, this paper poposes a 
trajectory controller network (TCN) Echnique lasedon linsar approximation 
model (LAVI) technique. In a TChJeadi coitroller cat be ofa simple form, 
which may be (btained straightforwardly via classical design a evolutionary 
means. Toco-oriinatetheoverall coitrollerperformance.the scheduling ofhe 
TCN is evolved through the entire operating envelope. Snce phnt step 
response dtaare ofEn eadily available in engineering pactice, the ctsignof 

such TCN B fully automated iBinganevolutionaryalgorithmwithoutthe teed 
of model identification. This is liustrated and validated though a ronlinear 
control ecample. 



1 Introduction 

A dynamic engineering system is usually nonlinear and complex in practice. Plant 
dynamics may vary significantly with changes of operating conditions. Therefore, the 
use of a single nominal linear model under one operating condition, and hence 
controllers designed out of such a plant model, are often unreliable and inadequate to 
represent a practical system. The recently developed local controller network 
techniques [5], have provided some effective solutions to these problems, but they are 
based on locally linearised models. 

To address these problems more completely for a wider range of operating 
trajectories and to make use of plant step-response data that are often readily available 
in engineering practice, this paper proposes a trajectory controller network (TCN) 
technique based on linear approximation model (LAM) technique [2]. Such a LAM 
network is obtainable directly from plant step-response by fitting nonlinear 
trajectories between two operating levels. As preliminaries to design, this modelling 
technique is outlined in Section 2. 

In a TCN, each controller can be of a simple form, such as a proportional plus 
integral plus derivative (PID) controller, which may be obtained straightforwardly via 
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classical design or evolutionary means. To co-ordinate the overall controller 
performance, the scheduling of the TCN is evolved through the entire operating 
envelope. This is detailed in Section 3. Section 4 illustrates and validates the TCN 
technique through a nonlinear control example. Finally, conclusions are drawn in 
Section 5. 



2 Linear Approximation Model for Nonlinear System Modelling 

Here, the LAM to approximate a nonlinear plant is illustrated through an example. 
The plant used for the example is a twin-tank coupled nonlinear hydraulic system that 
models liquid-level found in chemical and diary plants. The scaled down model can 
also be found in the laboratory. Based on the Bernoulli’s mass-balance and flow 
equations, the system structure is given by: 



h,~ 




- sgnf/zj - - /!2 


+ 


a Q 
A 


V' 


_^2_ 




sgnfhi h^) h^\ ^^pg{h^ Hq) 




0 0 


0 



The system input is the voltage applied to the pump, v,, and the system output is the 
liquid level in tank 2, h^. The coefficients of the twin tank are tabulated in Table 1. 
The non-linearity of the plant model is clearly plotted as shown in Fig. 1. 

Table 1. Nonlinear system parameters 



Height of water in tank 1 


/jj(m) 


Height of water in tank 2 


/r,(m) 


minimum height of water in tank 


H„=0.03 m 


Cross sectional area of tank 1&2 


A =0.01 m" 


Discharge coefficient of orifice 1 


c, = 0.53 


Discharge coefficient of orifice 2 


Cj = 0.63 


Cross sectional area of orifice 1 


a, = 0.0000396 m" 


Cross sectional area of orifice 2 


= 0.0000386 m 


Gravitational constant 


g = 9.81 m s'^ 


Per- volt Pump Flow rate 


a = 0.000007 (m' s ' V') 


Flow rate from tank 1 to tank 2 


a (m's ‘) 


Discharge rate 


a(m's') 
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Fig. 1. Non-linearity of the plant, is the steady state liquid level of tank 2 when the input 
voltage V. is applied to the pump 

If three operating points on the nonlinear trajectory are used, they can he chosen 
based on the rate of change or equally divided from the full operating range of the 
liquid level. The simple division is used because of the trajectory capability of the 
LAM models. The three operating points are 0.05m, 0.1m and 0.15m. Generated step 
responses from the LAM at these points are shown in Fig. 2. 




Fig. 2. Step responses at three operating points of a LAM network 
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3 Evolving a TCN 

Many control system design methods are based on linear systems analysis. Here, a 
simple PID control system is design for a LAM. To apply the TCN technique to 
nonlinear plants, controllers must be designed for the entire LAM network of the 
nonlinear system. 

This is to provide adequate performance across the operating envelope of the 
system. Three PID controllers in the TCN are scheduled or switch between them as 
shown in Fig. 3. During controller operation, a variable indicating operating point is 
monitored and different controllers (or controller parameters) are activated according 
to this scheduling variable. In this design, the plant output y(t) is used as scheduling 
variable to schedule the output of the controllers. 



Scheduling variable 




The TCN uses a linear interpolation or weighting schedule as shown in Fig. 4. 



Weight 

100% 




Fig. 4. A simple interpolation schedule in forming a TCN 
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Therefore, at any output level y(t), the individual controller outputs uft) are 
interpolated giving a final controlling output u( t) using equation (2), where Pj=0.05tn, 
P=Q.\m and P3=0.15m. 






Pi.i-Pi 



Pi.i-P 



u(t) = • 



ifP,<y(t)<P^,, 

if yit) < Pi 

ify(t)>Pn 



where i=l,...,n-l and n is the total number of linear controller 



( 2 ) 



Here, interpolation may also applied to the controller parameters K^, K. and K^. 



4 Design Example and Validation 



4.1 Generating Trajectory Controllers from Step Responses 

Individual PID controllers from a step-response trajectory to each of the three 
operating points are generated from the PIDeasy’''’^ design automation package [4], as 
shown in Fig. 5. 



PlDeasw<TM> - Pluci & Plav Oatinal PID Genei'atoi' 



Pile Method Design Options Help 

rB^TE^ESPONS^flT^^B^lIfiM^fiRfinETERS 

Specify the plant &tep~response data filenane: 
Specify the excitation step size applied: 
Specify tlie sampling period used <sec>: 



SET PID ON-LINE 

[ plant2.dat 
15.0 
[0.005 



Uiew Digital PID -41 



ftlt*RED hey for Cominand 




Actuator Upper limit: [le20 I Lower limit: [~le20 

[ ) Add Anti-windup <Not always necessary for a good PID> 

To manually set or improve your otm PID settings from: 

Kp <->Ki or < >Ti <->Kd or 

1 1106 ] 2 [190 1 3 [0.5 

To save all siruilated tine-responses in: 

To save all simulated frequency-responses in: 

To save generated digital PID in C file: 

To save generated digital PID in «lava File: 



Uiew Performance 
Alt*F4=Close window 



FI =Help 



Fig. 5. Direct design from plant response using PIDeasy'’^^ 



PIDeasyTM analyses step response data and generates an appropriate PID controller 
from them. At each operating point, fitting the step response generated from a LAM 
produces fast generation of PID controller. The closed loop responses at these 
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operating points are plotted on the same graph shown in Fig. 6. Note that the fast 
generation of PID controllers using linear PIDeasyf^ technique is tested against the 
nonlinear plant. This reveals the need of network tuning. 



Performance of each individual trajectory PID controller 




Fig. 6. Performance of each Individual trajectory PID controller 



4.2. Networking Through Evolution 

A genetic algorithm (GA) provides globally optimal solutions to engineering design 
problems by emulating natural evolution. A population of potential solutions evolves 
using the evolutionary operators of crossover, mutation and selection to approach 
optimal solutions. One advantage of a GA for optimisation is that the objective or 
fitness function needs not to be differentiable. Here, the objective function to be 
minimised is the summation of all errors across the entire TCN at n reference 
operating points within a given time period m. 

n m 

J = t i \e(t)\ 

ref=l t=0 ( 3 ) 

Here, there are n=5 reference levels used to evaluate the error tracking performance as 
shown in Fig. 7. The 5 reference levels are set to 0.05m, 0.075m, 0.1m, 0.125m and 
0.15m. These 5 points cover the whole trajectory and including two unseen operating 
points. Each reference is tested for a period of m=1000 sec, where 

e(t)=\ r(t)-y(t) I. 



represents the tracking error between the closed-loop output y(t) and the command 
r(t). 
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Fig. 7. Evaluation points in the operating envelope. 

To evolve TCN based on the fast generated PID controllers out of LAM, all the 
parameters of the three linear controllers and the scheduling weights, are evolved 
simultaneously, at the seen and unseen operating points along in operating envelope. 
The closed loop responses of the finally evolved TCN at the end of 50 generation with 
a population size of 50 are shown in Fig. 8 for all of the tested operating levels 
depicted in Fig. 7. It can be seen that the linear TCN provided an excellent solution to 
the nonlinear control problem. 



Closed loop responses of TCN 




Fig. 8. Closed loop responses of the TCN at operating points including the unseen ones at 
0.125m and 0.075m. 
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5 Discussion and Conclusion 

To assist control system design for a wide range of operating envelope for nonlinear 
plants, this paper has developed a trajectory controller network (TCN) technique 
based on linear approximation model (LAM) technique. The example shows that the 
linear TCN used to control a nonlinear system performs well in the entire operating 
envelope. This offers potential benefits and simplicity for control of nonlinear 
systems. The results show that the GA based automatic controller network design for 
nonlinear systems is possible and useful. Such a network is easily designed from 
sampled response data. 
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Abstract. An algorithmic parameter tuning methodology for controller design 
of complex systems is needed. This methodology should offer designers a great 
degree of flexibility and give insight into the potentials of the controller 
structure and the consequences of the design decisions that are made. Such a 
method is proposed here. For an exploratory phase a new pareto-ranked genetic 
algorithm is proposed to generate an evenly dispersed set of near optimal, 
global, solutions. By pair-wise preference statements on design alternatives a 
linear program is set up as a formal means for selecting the solution with best 
overall designer satisfaction. In a following interactive design phase using 
nonlinear programming techniques with a priori decisions on allowed quality 
levels, a best tuning compromise in competing requirements satisfaction is 
searched for while guaranteeing pareto-optimality. In particular, this two-phase 
tuning approach allows the designer to balance nominal control performance 
and multi-model control robustness. 



1 Introduction 

Control engineering work is mainly occupied with adapting a control system 
architecture with given control law structure, sensors and actuators, to needs of 
changed product requirements or new product versions. This is called ‘incremental 
design’. In practice this occurs much more often than starting control system design 
afresh. The essence of incremental design is adaptation hy tuning the control law 
parameters, partial replacement or augmentation of the control law structure by 
dynamic compensators, filters, and signal limiters, and tuning the overall structure in 
concurrence with the basic control law parameters. 

Common industrial practice is hardware-in-the-loop, manual, tuning called 
‘calibrating’. However, since there may be very many parameters to be tuned manual 
tuning is not efficient neither in required engineering costs nor in exploiting the full 
potential of the chosen system architecture with respect to multivariate requirements. 
Therefore, ‘virtual product engineering’ based on high fidelity system model 
simulations is more and more becoming the engineering life style of choice. CACSD, 
i.e.. Computer Automated Control System Design, is the discipline to provide the 
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pertinent technology in control engineering, making use of recent advances in 
automated symbolical / numerical system dynamics modeling and algorithmic 
parameter search via nonlinear programming or evolutionary computation. In 
particular, CACSD supports the design computation loop depicted in Figure 1, cf. [1]. 




pareto- 

optimal 

algorith. 

tuning 



requirements 

capture 



quaiity functions 
characteristics 
design directors 



T tuner parameters 
K controiier parameters 
L, H quaiity scaiings 



Fig. 1. CACSD generic computation loop to support incremental control design 

This CACSD computation loop is generic in that it allows to incorporate any 
controller structure to be tuned. Tuning parameters T may relate to controller 
parameters either directly, K = T, e.g., in PID control [2], or indirectly via an analytic 
synthesis method, K = f(T, synthesis model), e.g., in [3] control, as well as fuzzy 
control, K = f(T, fuzzy rules), where in the latter case possible tuning parameters are 
the scaling factors of the membership functions and the weighting factors of the fuzzy 
control rules [4]. Furthermore, it allows to apply any analysis method, e.g., linear 
methods in state space and frequency domain as well as non-linear time simulation, to 
cope with non-commensurable control quality evaluations simultaneously. 

To assess requirements satisfaction, design requirements are captured formally by 
quality functions. The fuzzy-type interval formulation of quality functions, cf. Section 
2, allows to deal with quality levels, e.g., levels of ‘good’, ‘acceptable’, ‘bad’, 
requirement satisfaction. The data obtained by evaluation of all the quality functions 
feed a tuning algorithm to compute pareto-optimal tuning parameter values. For this 
kind of data-driven tuning both evolutionary algorithms, cf. Section 3, or non-linear 
programming algorithms, cf. Section 4, can be used. 

Pareto-optimality lends itself not to a unique solution. Therefore, in Section 3.1 a 
new multi-objective genetic algorithm is proposed, which has the special property that 
it yields evenly dispersed solutions in or near to the pareto-optimal set, thus making 
best use of evolutionary computation to produce a rich set of design alternatives. 
Having many alternatives available to choose from, selection of the best candidate 
needs to follow a formal approach. This is dealt with in Section 3.2. 

Nonlinear programming formulations in Section 4 are used to generate dedicated 
pareto-optimal design alternatives either to attain optimal designer satisfaction or to 
iterate quantitative compromises in competing requirements satisfaction. 
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This suggests a two-phase tuning procedure to achieve multi-model control 
robustness, Section 5. In phase one a global multi-objective design for a nominal plant 
model instantiation is carried out using evolutionary computation, and in phase two 
interactive nonlinear programming computations are applied to compromise nominal 
control behavior with off-nominal behavior characterized by a number of off-nominal 
plant model instantiations. By this tuning off-nominal behavior is to become at least 
‘acceptable’ while nominal behavior is to be kept within the ‘good’ quality level. 



2 Requirements Capture and Satisfaction Assessment 

For design assessment design characteristics like system damping, steady state error, 
gain and phase margins, or maximum control rate, have to be transformed into a 
quality value which indicates the degree to which requirements are met. Two kinds of 
mathematical formulations are commonly in use: positive definite ‘the-smaller-the- 
better’ functionals of time and frequency responses, e.g., [2], which ought to be 
minimized, and inequalities on the design characteristics, which ought to be satisfied 
as constraints, e.g., [5], [3]. 

Advantages of the two approaches for quality modeling can be combined by the- 
smaller-the-better interval quality functions, where requirement satisfaction is 
considered ‘good’ for one range with function value zero, ‘acceptable’ in a range with 
function value not greater than one, and ‘bad’ outside a limiting range. 

Such an interval quality function, q(c), is mathematically defined on the design 
characteristics, c, by the max-operator (1) with four interval values b/ < g/ < gi, < b/, 
compliant with ‘bad’, ‘acceptable’ and ‘good’ characteristics values, cf. Figure 2, 

q(c) = max{Z(c), 0, 

L(c) =(c-g,)/(b,-g,), b,<g, 

H(c) = (c-gJI(b^-gJ, g,<gh<b^. 

Requirement satisfaction is assessed as the better, the smaller a value this quality 
function assumes. Furthermore, the max-formulation fits to fuzzy logic AND- 
operation with max-operator [7] to make overall ‘good’, ‘acceptable’, ‘bad’ system 
quality statements in the vein of fuzzy logic. It also allows to combine an enumerated 

set of commensurable quality characteristics Cj^, Z^(Cj(.), , to form a 

compound quality function for, e.g., taking account of all eigenvalues concurrently or 
handling all values := c(f^ ) of a discretized time response as an entity in 

requirements capture for robust tracking thumbprint performance [6]. With H:=0, the 
‘good’ interval is open to the right, see the example of Figure 2, with L:=0 it is open 
to the left. 
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Fig. 2. Quality function for eigenvalue damping, with ^ > 0.7 ‘good’ and > 0.3 ‘acceptable’ 



3 Pareto-Tuning by Genetic Algorithms and Design Preference 

Control systems parameter tuning is always a multi-objective problem with 
competing requirements of control performance versus control effort and robustness. 
Hence the designer’s prime task is to search for a suitable tradeoff while generating 
feasible solutions. This search ought to be confined to the set of ‘best achievable’ 
compromise solutions known as pareto-optimal solutions. Generally, a design 
alternative a\ is said to be pareto preferred to an alternative aj if all quality measures q 
of a\ are better (smaller) than or equal to those of aj, with at least one being strictly 
better. Thus, a pareto-optimal, or non-dominated, solution is one where no quality 
measure can be improved without causing degradation of at least one other quality 
measure. 

To make an informed tradeoff decision, the designer needs a rich set of design 
alternatives as well as formal methods to support a systematic selection process. 

Such a methodology is proposed here. A new pareto-ranked genetic algorithm 
generates an evenly dispersed set of design alternatives of near optimal solutions, 
giving the designer a global overview of what can be accomplished with the used 
controller structure, and by the method of preference-directed design that solution, 
which results in greatest overall designer satisfaction, is selected. 



3.1 A New Genetic Algorithm to Generate an Evenly Dispersed Set of Solutions 

A natural way of finding the pareto set by genetic algorithms was proposed by 
Fonseca and Fleming [8]. This approach measures the fitness of an individual by the 
number of other individuals that dominate it in the sense of pareto preference. 
Accordingly, a population is ranked, where the best solutions will be the non- 
dominated ones. Thus, the non-dominated solutions will always be most likely to be 
selected, leading to a convergence of the population to the pareto set. 

Figure 3 gives an example of how a population would be ranked with this 
algorithm, where minimization of two quality measures Ci, C 2 is the goal. In this 
example, there are three non-dominated solutions which are ranked with a zero; all 
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other solutions are dominated and their rank is determined by how many other 
solutions are better in the pareto sense. 




Fig. 3. Pareto ranking according to Fonseca and Fleming [8] 



However, due to genetic drift, the population tends to converge to a single point on 
the pareto surface. Since it is the aim of an exploratory design phase to provide the 
designer with a rich basis of alternatives for possible tradeoff decisions, finding only 
one pareto solution is unsatisfactory. Thus, genetic drift should be avoided, which can 
be accomplished by methods like multiple sub-populations or penalizing overcrowded 
neighborhoods (fitness sharing) [3]. 

A variant of fitness sharing is proposed here for the purpose of finding an evenly 
dispersed near-pareto-optimal set of solutions. The idea is to incorporate the 
proximity of other individuals into the fitness of one individual such that individuals 
in remote regions of the search space will enjoy an advantage over those in more 
overcrowded regions. Since some individuals in the crowded regions should remain, 
so as to not distract the genetic algorithm from promising regions of the search space, 
penalties for crowding are limited to values less than 1. As the fitness score from 
pareto ranking is an integer, this ensures that the individuals are always ranked first 
by pareto domination but among individuals of the same pareto rank, the individuals 
are forced to spread out evenly. 

Having mapped the n criteria values to [0,1] by c/ = (c/ -Ci^miny(ci,max-Ci,min) the 
penalty to attain an evenly dispersed set is computed by the following formula: 

p = (l-S)- I X 

where b is chosen small but greater than zero, e.g., b = 0. 1 , c,j signifies the 
criterion value of the individual, 'current’ is the individual for which the penalty is 
calculated and ’closest’ is the individual which is the closest preceding the current one 
in a list of all individuals sorted by pareto-rank. This list is sorted in ascending order, 
so that the best individuals come first, and between two individuals of the same rank, 
the position in the list is decided by random. 

Thus, the penalty is always smaller than 1 and the adjusted fitness value is always 
strictly positive, which is necessary for some genetic search algorithms. 

This variant of a multiobjective genetic algorithm has been implemented by means 
of the Genetic and Evolutionary Algorithm Toolbox [9]. Results show that this 
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ranking procedure assures convergence towards the entire pareto set as opposed to 
either converging to just a part of it or favoring remote areas of the search space with 
less than optimal criteria values. The evolutionary optimization is carried out until 
either a certain number of generations is reached or a liveliness criterion is met. But 
since no analytical convergence criterion is used there is no guarantee for obtaining 
exact optimal solutions rather than attaining a near pareto-optimal set. 

One example of how this multiobjective genetic algorithm fares with penalties (2) 
can be seen in Figure 4. The optimization results are shown together with the 
boundary of the set of all possible solutions. As measure for the degree of spread 
along the pareto surface the average standard deviation in the obtained pareto value 
set, computed over 1000 runs and scaled to a maximum of 1, is shown in Table 1. 



Table 1. Average standard deviation in computed pareto value set 



Without Penalties 


With Penalties 


Average Standard Deviation 0.1493 


0.2417 



All Generations 

4000 
3000 
2000 
1000 
0 

1 2 3 4 5 6 7 8 

4000 
3000 
2000 
1000 
0 

1 2 3 4 5 6 7 8 

Fig. 4. Example: convergence to evenly dispersed pareto-optimal value set 
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3.2 Design Selection for Best Designer Satisfaction 

After an evenly dispersed set of solutions is found, that solution which gives highest 
designer satisfaction is to be selected. Commonly, designer satisfaction is measured 
by a normalized weighted-sum value function v(q) and a design is judged the better 
the smaller the value v{q): 



V = '^w ■ q (T) , with: X = 1, VA: : ^ 0 ■ 

k 



( 3 ) 
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The problem, however, is how to attribute a priori the numerical weights Wk in 
compliance with designer’s intentions to formally decide on the ‘best’ solution with 
minimal value v(q). The a posteriori approach of preference-directed design selection 
[10] copes with this problem by so-called imprecise value functions based on a 
number of pair-wise preference statements to be made by the designer. 

If a designer makes a preference statement “Tuning 7) is superior to Tf then by (3) 
this implies 






(4) 



which can be rewritten as 

=l,Vi:w, >0, 



(5) 



With known qik, qjk, a set of preference statements (5) describes a subspace for 
admissible weights Wk- In [10] this constitutes an ‘imprecise value function’ to check 
whether a compatible preference ordering exists among other design alternatives as 
well. This is formally decided by solvability of the following linear program: 



min ^ ) ^2*1 j{ ('^2*1 ) 



( 6 ) 



s.t. 



)-«»(?’, ))>o 



X 2*1-1, k (^2*/-l ) ^2*1, k (^2*/ ^ ^ 

k 

Y,Wk=iyk-.Wk>0 , 



where for j = (2*1-1) and i = (2*1) the quality function values q2*l-l,k, ^l2*l,k are those 
of a pair of additional inferior and superior design alternatives, respectively. 

Based on a few compatible preference statements (5) this formalism allows to 
partially order the set of design alternatives for selecting ‘the best’ one. The imprecise 
value function is a means to prune inferior paths during design space search thus 
reducing the complexity of the selection process. Having finally selected the best 
available design alternative, a compatible set of weights can be computed by (6), 
minimizing the difference of the value function of the chosen alternative to its ideal, 
i.e. superior, value of designer satisfaction, which is characterized by qk = 0. 
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4 Pareto-Tuning by Interactive Nonlinear Programming 

The genetic algorithm with pareto preferred ranking, as proposed in Section 3.1, fits 
well to the nature of evolutionary computation since on return to the many function 
evaluations that evolutionary computation requires it yields a rich set of solutions 
evenly dispersed in or close to the entire pareto-optimal set. Furthermore, genetic 
algorithms cope well with a large number of parameters and with a large search space, 
which makes them likely to find the global instead of a local solution in multimodal 
problems. Together with formal decision support, as dealt with in Section 3.2, this is 
well suited for selecting a posteriori a design candidate with best designer satisfaction 
among a number of global design alternatives. 

On the other side, nonlinear programming approaches to attain pareto-optimal 
solutions are based on an analytical optimality condition, which makes them very 
efficient to compute just one, a priori dedicated, pareto-optimal solution in the local 
neighborhood of where the algorithms gets started. Furthermore, the necessary 
Karush-Kuhn-Tucker optimality conditions yield a numerical convergence condition 
that allows to attain a pareto optimum with high accuracy. This makes nonlinear 
programming algorithms suitable for ‘fine tuning’. If parameterized in a decision 
intuitive way, interactive, declarative search to attain a specific, ‘best’, compromising 
solution for requirement satisfaction under local design conflicts becomes feasible. 

Nonlinear Programming can be used to compute pareto-optimal tuning values Thy 
solving [11] the normalized weighted- sum minimization problem, cf. (3) 

min X , s.t. = \yk \w^>0,q>0, ( 7 ) 

‘ k k 

or the min-max optimization problem 

min max{^, (r )/r/, }, s.t. 0<q^(;r)<d k = {i,j] . (8) 

T / 

Optimization (7) or (8) is a sufficient condition to yield a pareto-optimal solution 
with parameterized specific properties. There are standard algorithms of nonlinear 
programming, like SQP, to be used starting with a (global) solution by Section 3.1. 

Formulation (7) yields the solution of optimal designer satisfaction with respect to 
attributed weights as found, e.g., by the a posteriori decision procedure of Section 3.2. 
A priori selection of weights is not decision intuitive [11] to attain specific properties. 

Formulation (8) yields a pareto-optimal solution depending on parameters d. This 
formulation is well suited for compromising competing requirements by using these 
parameters iteratively as a priori ‘design directors’ to balance requirements 
satisfaction within a feasible solution set. A decision intuitive approach to choose 
design directors d is now proposed, which is inspired by the performance vector 
decision systematics due to Kreisselmeier [12]. 

Initialization Step: Start by a global solution with designer satisfaction according to 
Section 3 and pareto-optimize this solution by solving the unconstrained min-max 
problem with di = {lj, j e0. This yields a balanced solution where all quality 
functions get pareto-minimal and the value function for designer satisfaction is further 
improved if the start solution is not yet an optimized one. 
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By solving the unconstrained min-max problem, the question for the main conflicts 
in requirements satisfaction can be answered: If T = 7’* is a minimizer, then 

max {q, {T*)} = q], = q]^ =d >Q, 

which means that the values of quality functions qd^^icii--- belonging to the most 
competing requirements are equal and that they have the largest value among all 
quality functions. Moreover, OC < 1 characterizes a feasible (‘acceptable’) solution, 
which gives room for compromising the most competing requirements within the set 
of pareto-optimal alternatives. 

Iterative Compromising Steps: Starting with a pareto-optimal solution, satisfaction 
of competing requirements cannot be improved simultaneously. This means that 
lowering the value of one quality function, qci(T), can be achieved only at the 
expense of a higher value of another, qc 2 (T), and vice versa. Different compromise 
solutions qc / versus qc 2 can be achieved by different choices of the design directors d 
in an iterative procedure.- 

With a given pareto-optimal solution ~ 

iteration step decide which of the most conflicting quality functions lqci, qc2j shall be 
improved, say qct- This choice may be made dependent on the weights that are 
associated to these quality functions via the formalism of Section 3. Then, 

concatenate E. {i,cl}, and choose dj^ = 1 and d^2 that ^*2 ^ ^c2 

(<1). Solving the constrained min-max problem, 

min max{^f ^ (T )} s.t. q^^ (T ) < d^^^’ > 

1 k 

then attains the best possible solution in the sense that all quality functions of interest 
are minimized up to the constraint of the quantified limit of degradation one declares 
to be acceptable for the main conflicting quality function. Thus in an iterative 
procedure one can search for a ‘best’ compromise satisfaction of competing 
requirements. 

Iterative compromising is best carried out in an interactive mode of working which 
needs fast algorithms to execute the CACSD computation loop of Figure 1. In 
addition it needs visual decision support on various information levels to best grasp 
design problem complexity. In particular, a graphical user interface [13] with a 
parallel coordinates display of the many quality functions, used as interactive steering 
aid to detect compromise conflicts and to choose design directors at runtime, greatly 
enhances engineering productivity. ANDECS_MOPS is such an environment [14]. 



5 Pareto-Optimal Multi-model Robustness Tuning 

Feedback control suffers from potential stability problems, but if properly designed 
feedback reduces parameter sensitivity. Therefore design of controllers, which are 
stability and performance robust with respect to off-nominal operation, is of prime 
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concern. Analytical robust-control theory, like |d-synthesis, relies on analytical 
stability criteria and pertinent (linear) plant model and uncertainty descriptions. Thus, 
it is restricted to problems with specific, commensurable, performance measures. 

A completely general approach to robust control design is so-called multi-model 
design as implied by the CACSD tuning loop, cf. Figure 1 . It is applicable to any kind 
of (non-linear) plant models and non-commensurable performance measures since it 
relies only on the data of the performance measures and not on their analytical 
description. Structural independence makes this kind of robust control approach 
applicable to any type of controller, i.e., PID, observer feedback, fuzzy control, etc. 

The idea of robust multi-model design is to state the design problem for a nominal 
plant model instantiation reflecting nominal operation conditions and nominal system 
parameters within pertinent tolerance bands. Then, the same problem is stated for a 
number of off-nominal model instantiations reflecting worst case plant behavior, e.g., 
fast, lightly damped, and slow, over-damped, behavior within the range of assumed 
operation conditions and parameter uncertainty intervals. The quality functions of all 
these formulations are concatenated to a single multiobjective problem for which a 
satisfying pareto-optimal solution is to be found. This approach is highly competitive 
in comparison to other (analytic) robust control approaches [15]. 

The solution approaches that are dealt with in Sections 2, 3, 4, are particularly 
suitable to be combined for this type of multi-model, multi-objective, robust control 
design in form of a two-phase design procedure : 

In phase 1, only a nominal model instantiation is considered and interval quality 
functions formulated according to Section 2 are optimized by the multiobjective 
genetic algorithm of Section 3.1 to yield a rich set of global, pareto-optimal, design 
alternatives as basis for designer preference selection. Section 3.2. 

In phase 2, the ‘best’ nominal performance achieved in phase 1 is embedded in 
‘good’ intervals by re-scaling the quality levels as required. This solution is used to 
start further tuning under the aspect of robustness: The off-nominal design cases are 
added to the nominal case and interactive nonlinear programming iterations according 
to Section 4 are simultaneously applied to all design cases to robustify the result of 
the first, nominal, design phase. 

Thus, the user is allowed to make quantitative trade-off decisions concerning 
nominal versus robust performance. In these decisions off-nominal control behavior, 
characterized by worst-case plant model instantiations, should become at least 
‘acceptable’, while nominal behavior is to be kept within a ‘good’ quality level. 



6 Conclusion 

A parameter tuning methodology to support control design automation is described. It 
uses a multiobjective genetic algorithm with fitness sharing to find a rich set of global 
solutions evenly dispersed in or near to the pareto-optimal set, from which a design 
candidate for best designer satisfaction is formally selected via pair-wise preference 
statements. Then min-max nonlinear programming is applied for compromise tuning 
to attain a pareto-optimal solution with best tradeoffs in requirements satisfaction. An 
on-line interactive mode of working using nonlinear programming in the 
compromising phase is supported by a systematics for choosing ‘design directors’ as 
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allowable upper bounds to limit the expense one is willing to pay in making tradeoffs. 
Together with requirements capture by fuzzy-type interval quality functions, this two- 
phase approach is well suited to quantitative multi-model-robust control design with 
non-commensurable performance and robustness requirements. 
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Abstract. 

Currently there exist various cost-assignment schemes that perform the necessary 
scalarization of the objective values when applied to a multi-objective optimization 
problem. Of course, the final decision depends highly on the nature of the problem 
but given the multiplicity of the schemes combined with the fact that what the user 
ultimately needs is a single compromise solution it is evident that elaborating the 
selection of the method is not a trivial task. This paper intends to address this problem 
by extending the benchmarks of optimality and reach time given in [1] to 
mutliobjective optimization problems. A number of existing cost-assignment schemes 
are evaluated using such benchmarks. 



1. Introduction 

Having in mind the number of existing approaches to cost-assignment one could 
presume that the next step would be an appropriate choice. The concept of Pareto 
dominance has proven to be a great aid towards the formulation of the various 
schemes but further thinking reveals that what the user would like to have is simply a 
single compromise solution and not all of the solutions that form the Pareto-optimal 
set. As stated in [2], “although a Pareto-optimal solution should always be a better 
compromise solution than any solution it dominates, not all Pareto-optimal solutions 
may constitute acceptable compromise solutions”. Therefore, what is needed is a 
performance index that can be used for the evaluation of the suitability of each 
scheme in the context of a specific problem. 

To address this issue, two benchmarks used presented in Section 2. Section 3 
describes a benchmark problem used for the comparison. Section 4 presents the 
evolutionary algorithm employed along with an outline of the various cost-assignment 
methods. Comparison is made between a number of evolutionary algorithms in 
Section 5. Conclusions are drawn in section 6. 



2. The Benchmarks 

Two benchmarks used for the evaluation of the different approaches have been 
defined in [1]. In this section they are going to be briefly presented. 
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The first benchmark is called ‘optimality’. Suppose that we have a test function 
f(x): F, where X R", F R“, 

where ^ represents the search spac^i n dimensions, 

R"fepresents the space of all the possible objective values, 
n is the number of parameters, 
m is the number of the objectives 
f G F collection of the individual objective elements 

Also, consider the theoretical objective vector fo = { f(x) } that contains the 
objective values that can ultimately be reached. Finally, consider an objective reached 
as in Eq. 1 . 



/ 



( - ^ 



V J 






( 1 ) 



with ^Xo, representing a corresponding solution found. 

The optimality measures how close an objective reached is to the theoretical 
objective vector and is calculated using the formula in Eq.2. 



fo-fo 

Optimalit^ f^ \ = 1 - ^ — s [0,1] 

V Ja -f_f 



(2) 



where 'f and f. are the upper and lower bounds of f respectively. 

Any norm can be used to evaluate the optimality of an objective and this paper 
uses the Euclidean metric (a=2) for this purpose. 

The above formula needs to be refined when the problem addressed is a non- 
dominant or non-commensurate one since no such concept as ‘overall optimality’ can 
be assessed in a problem of this kind. Since this is the case for this paper, the 
‘distance to demands’ method explained in [3], is used here. 

The second benchmark this short study uses is one that measures the convergence 
of the algorithm and is called ‘reach time’. The reach time is defined as the total 
number of function evaluations performed by the algorithm by which the optimality 
of the best individual first reaches b. 

Reach time\=C^ ( 3 ) 

\b 

Eor the purposes of the tests, b is set to 0.999, a certainly high value that may not 
always be reached by the algorithm. Because of that, a single algorithm terminates 
when either the set optimality threshold is reached or 20n generations of size 20nxm 
have been evolved. Those termination conditions are identical to the ones used in [1] 
with the latter one meaning that the algorithm is not supposed to perform worse than 
an O(n^) algorithm in terms of computational time. 
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3. The Problem 

A set of two objective functions (Fonsceca and Fleming, 1995) was chosen for the 
evaluation of the cost-schemes. The functions of Eq.4 and Eq.5 were chosen in an 
effort to produce as “standard” a Pareto optimal front as possible. 




Each individual consists of a real-valued vector of n parameters. For the purposes 
of this paper n was set to the value of 2, with each parameter coded in the interval [-2, 
2). 

The individuals that form the Pareto-optimal set belong on the line shown in Fig. 1 . 
Functions fl and f2 are plotted for n=2 in figures 2 and 3 respectively. 




a Q1 03 03 Q4 0 5 oe or OB 0 9 I 

n 



Fig. 1. Pareto optimal front 




Fig. 2. f 1 plotted for n=2 Fig- 3. f2 plotted for n=2 
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4.The Algorithm 

4.1 Selection, Crossover and Mutation 

The evolutionary algorithm that was used is quite a simple and straightforward 
one. It uses a binary tournament selection scheme to form the mating pool of the 
individuals after, of course, the cost assignment procedure has taken place. Each 
individual in the mating pool then randomly mates with another one using arithmetic 
crossover since each individual consists of a real-valued vector. Arithmetic crossover, 
as described in [4], is a canonical intermediate recombination operator, which 
produces the i-th component of the offspring by averaging it with some weight as 
defined in Eq.6. 

x[ = oocj,. + (l - a )x 2 i (6) 

Next, the offspring are evaluated and then refined by the simulated annealing (SA) 
technique. The SA positive mutation cycles where conducted using a non-linear 
Boltzmann learning schedule as the one employed in [5]. For the purposes of 
mutation, the creep mutation operator (Davis, 1989) was employed. As suggested in 
[6], entrapment must be alleviated in the case of this operator when used with a 
bounded small random amount for mutation. As such is the case here, entrapment can 
be said to have been partially overcome by the probabilistic nature of the SA 
technique, which maintains a probability of retaining lesser-valued individuals. 

After SA, the parents are merged with the offspring the new population is formed 
with binary tournament selection. 

The parameter settings of the algorithm, most of which are suggested in [5], are 
listed in table 1. The optimality threshold was set to a high value as the tests were 
intended to prohibit the algorithm to reach it so that a clearer picture of each scheme’s 
behavior could be obtained. 



Table 1. Parameter settings 



Optimality threshold 


0.999 


Number of parameters 


2 


Weight vector 


{ 1,1 } 


Priority vector 


{ 1, 0=top ) 


Goal vector 


{ 0.0, 0.0 ) 


Number of generations 


160 


Population size 


320 


Arithmetic crossover 


0.5 


constant 




Creep mutation 


0.05 


probability 




Tournament size 


2 


Boltzmann constant 


5E-06 


Initial temperature 


1E05 


Final temperature 


1 


Initial annealing factor 


0.3 


Transient constant 


16 
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4.2 The Cost- Assignment Schemes 

Each scheme is successively described below with a minimization problem 
assumed. 

4.2.1 The Weighted Sum Method 

According to this approach, all of the objectives are weighted by positive 
coeffieients defined by the user and are added together to obtain the cost. 



f{x) = '^wj,{x) 

k=l 

, where O denotes the cost assignment scheme and x is the parameter vector. 

It must be noted that the same weights that are used here are also used to weight 
the objective vectors prior to the caleulation of the norms of the optimality benchmark 
measure as suggested in [1]. 



4.2.2 The Minimax Method 



This method tries to minimize the maximum weighted difference between the 
objeetives and the goals, with the weights and the goals supplied by the user. 



f{x) 






: max 

k=\...m 



Sk 






(8) 



4.2.3 The Target Vector Method 

This approach minimizes the distance of the objective vector from the goal veetor 
using a defined distanee measure. Again the user supplies the goals. The Euelidean 
metrie was used as the distance measure in this case. 

/W=||[/W-^]h^-1 



4.2.4 The Lexicographic Method 

Here, the objectives are assigned distinct priorities and the selection proceeds with 
the comparison of the individuals with respeet to the objective of the highest priority 
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with any ties resolved by a successive comparison with respect to the objective with 
the second-highest priority, until the lowest priority objective is reached. 



9?" ^{0,1,...|X— 1},|J, = pop_size 

/ ) = L ^ A = 

where l(condition) evaluates to unity if condition is true and 
f{xj)e < 3pe {l...m} : 

^ke{p,---, m], [xj ) < (x,. ) a (x^. ) < 4 (x; ) 



4.2.5 Pareto Ranking (Goldberg’s Approach) 

According to the definition in [6] all non-dominated individuals are assigned a cost 
of one and then removed from contention with the next set of non-dominated 
individuals assigned a cost of two until the whole population has been ranked. 

<I> : 9?" ^ {l,...|i}, |i = pop _ size 
fi 1 /UJWye {l...|X} 

(^ <= not (/ (x,. )p < /(x,. )), V; G {1 . . . |X } 

\{/:$(/(x,))<(^} 

, where the p< condition denotes partial domination of the individual j over the 
individual i and is true if and only if 

VitG 

fk [xj ) ^ A (a ) A 3k G {1 . . . m} : (x^. ) < A (a ) (13) 



4.2.6 Pareto Ranking (Fonseca and Fleming’ s Approach) 

Proposed in 1993, this approach ranks an individual according to the number of 
individuals that dominate him. 

<I> : jK" ^ {0,1,... |X -1},|X = pop _size 



fix,) = Y^l /(xjp</(x,.) 
J=1 Iv 



^ . (14) 

p = pop_size 



J 




164 



K. Koukoulakis and Y. Li 



4.2.7 Pareto Ranking (With Goals and Priorities) 

This approach comhines the pareto-optimality concept with goal and priority 
information. Equal priorities may he assigned to different objectives with both the 
priorities and the goals supplied by the user. Individuals are compared as in the 
lexicographic method but it is also affected from whether the individuals attain the 
goals set or not. 

<I> :9t" — > (0,1,... p, -lj,p = pop _ size 
= jh = pop_size 

, where 

the condition within the brackets denotes preferrability of the j-th individual over 
the i-th individual and 

g is the preference vector, a vector that contains the goals of each objective 
grouped by priority 

As for the evaluation of the condition it is deemed too detailed to mention here but 
is fully described in [2] . 



5. Comparison Results 

For each method, 10 experiments were carried out each with a random initial 
population, with an experiment terminating either when the optimality threshold has 
been reached or 400mn^ generations have been evolved. A discussion of the results 
obtained follows. 

5.1 Pareto Front Sampling and Diversity 

A cost-assignment scheme is considered successful if it has managed to offer a 
diverse sample of the pareto-optimal front as quickly as possible. With this in mind, a 
short discussion for each scheme tested follows. 

A snapshot of the population in which the most optimal individual was found for 
each scheme can be seen at figures 4-10. Remember that the goals were set to 0.0 for 
both objectives. 

The weighted sum approach was unable to sample the concave region of the line, 
focusing entirely on the zero-cost line f 2 = -fi. 

With identical, equally weighted goals, the minimax scheme failed to sample the 
pareto front. Nevertheless, with appropriate goal and weight settings it can prove 
successful, but surely less successful than the pareto-based approach with goals and 
priorities, which provides a better sampling in a quick and more efficient manner 
using only the goal information available. 

The target vector scheme has introduced better sampling diversity than the 
minimax approach in roughly the same time. The diversity is even better than the 
pareto-based approach with goals and priorities but the latter scheme is significantly 
quicker in providing its results. 
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Fonseca and Fleming’s approach along with Goldberg’s original one has indeed 
most quickly sampled a very good proportion of the pareto-optimal front with the 
former being better at that. It must be noted that they have performed better at that 
than the last scheme without using any information available. This leads us to the 
conclusion that in the case of unattainable goals both of these schemes can offer a 
better sample of the front than the last scheme in some applications. 

The last approach has performed very well using both the goal and priority 
information. It is interesting to compare it with the lexicographic cost-assignment 
scheme, which only uses priorities. It can clearly be seen that the latter scheme has 
driven the population to the minimisation of f 2 , which has a higher priority over fi. 
So, it can be said that the lexicographic method needs an aid for better results and the 
most obvious one is niching combined with mating restriction. 

5.2 Optimality 

As far as optimality is concerned, as can be seen in Fig. 11, Goldberg’s and 
Fonseca and Fleming’s approaches have both quickly given optimal solutions without 
using the goal information and also have the added bonus of good diversity. Of course 
with attainable goals, the pareto-based approach with goals and priorities should be 
the quickest cost-assignment scheme to offer the most optimal solution. 

5.3 Reach Time 

As no method managed to reach the high optimality threshold of 0.999 (max = 
1.0), all of them had a reach time of 400mn^ = 51200. 



6. Conclusions 

The cost-assignment scheme acts as the driving force of the algorithm. Performing 
the scalarization of the objectives, it is the determining factor of evaluation. The 
purpose of this paper was to expose the magnitude of its impact on (a) the quality of 
the sampling of the Pareto-optimal front and (b) on the speed at which this quality is 
achieved. 

The goals were deliberately set unattainable because they were intended to be a 
means to push the population towards the front. The optimality benchmark itself 
worked in accord with the goal settings so that it could act as an observer of the 
algorithm’ s behaviour rather than a strict evaluator. 

It is thus concluded that for a given problem, the sampling of the Pareto-optimal 
front is generally easier achieved with the Pareto-based cost schemes. As for the rest 
of the schemes, it is thought that their usefulness can only be experienced with proper 
tuning of their associated parameters. 

As a further study, it would be interesting to test all of the schemes in the context 
of a harder problem, that is a problem with a more diverse front. Finally, it is believed 
that this testing should employ a wide range of weights so that the promising 
aggregating ‘target vector’ scheme can be examined more closely. 
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Fig. 10. Pareto ranking 
(goals and priorities), Gen. 
47 
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Fig. 11. Maximum optimality 
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Abstract 

This paper describes how the process of synthesizing the design of both the 
topology and the numerical parameter values (tuning) for a controller can 
be automated by using genetic programming. Genetic programming can be 
used to automatically make the decisions concerning the total number of 
signal processing blocks to be employed in a controller, the type of each 
block, the topological interconnections between the blocks, and the values 
of all parameters for all blocks requiring parameters. In synthesizing the 
design of controllers, genetic programming can simultaneously optimize 
prespecified performance metrics (such as minimizing the time required to 
bring the plant output to the desired value), satisfy time-domain constraints 
(such as overshoot and disturbance rejection), and satisfy frequency domain 
constraints. Evolutionary methods have the advantage of not being 
encumbered by preconceptions that limit its search to well-traveled paths. 
Genetic programming is applied to an illustrative problem involving the 
design of a controller for a three-lag plant with a significant (five- second) 
time delay in the external feedback from the plant to the controller. A delay 
in the feedback makes the design of an effective controller especially 
difficult. 

S. Cagnoni et al. (Eds.): EvoWorkshops 2000, LNCS 1803, pp. 168-177, 2000. 

© Springer- Verlag Berlin Heidelberg 2000 
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1 Introduction 

The process of creating (synthesizing) the design of a controller entails making 
decisions concerning the total number of processing blocks to be employed in the 
controller, the type of each signal processing block (e.g., lead, lag, gain, integrator, 
differentiator, adder, inverter, subtractor, and multiplier), the values of all parameters 
for all blocks requiring parameters, and the topological interconnections between the 
signal processing blocks. The latter includes the question of whether or not to employ 
internal feedback (i.e., feedback inside the controller). 

The problem of synthesizing a controller to satisfy prespecified requirements is 
sometimes solvable by analytic techniques (often oriented toward producing 
conventional PID controllers). However, as Boyd and Barratt stated in Linear 
Controller Design: Limits of Performance (1991), 

"The challenge for controller design is to productively use the enormous 
computing power available. Many current methods of computer-aided 
controller design simply automate procedures developed in the 1930’s 
through the 1950's ..." 

This paper describes how genetic programming can be used to automatically create 
both the topology and the numerical parameter values (i.e., the tuning) for a controller 
directly from a high-level statement of the requirements of the controller. Genetic 
programming can, if desired, simultaneously optimize prespecified performance 
metrics (such as minimizing the time required to bring the plant output to the desired 
value as measured by, say, the integral of the time-weighted absolute error), satisfy 
time-domain constraints (involving, say, overshoot and disturbance rejection), and 
satisfy frequency domain constraints. Evolutionary methods have the advantage of 
not being encumbered by preconceptions that limit their search to well-traveled paths. 

Section 2 describes an illustrative problem of controller synthesis. Section 3 
provides general background on genetic programming. Section 4 describes how 
genetic programming is applied to control problems. Section 5 describes the 
preparatory steps necessary to apply genetic programming to the illustrative control 
problem. Section 6 presents the results. 

2 Illustrative Problem 

The illustrative problem entails creation of both the topology and parameter values for 
a controller for a three-lag plant with a significant (five-second) time delay in the 
external feedback from the plant output to the controller such that plant output reaches 
the level of the reference signal in minimal time (as measured by the integral of the 
time- weighted absolute error), such that the overshoot in response to a step input is 
less than 2%, and such that the controller is robust in the face of disturbance (added 
into the controller output). The delay in the feedback makes the design of an effective 
controller especially difficult (Astrom and Hagglund 1995). The transfer function of 
the plant is 



Gis)^ 



Ke 



-5s 



{\ + TsY 

A controller presented in Astrom and Hagglund 1995 (page 225) delivers credible 
performance on this problem for values of A" = 1 and T = 1. 
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To make the problem more realistic, we added an additional constraint (satisfied 
by the controller presented in Astrom and Hagglund 1995) that the input to the plant is 
limited to the range between -40 and -h 40 volts. The plant in this paper operates over 
several different combinations of values for K and T (whereas the controller designed 
by Astrom and Hagglund was intended only for K=\ and T = 1). 

3 Background on Genetic Programming 

Genetic programming is an automatic technique for generating computer programs to 
solve, or approximately solve, problems. 

Genetic programming (Koza 1992; Koza and Rice 1992) is an extension of the 
genetic algorithm (Holland 1975). Genetic programming is capable (Koza 1994a, 
1994b) of evolving reusable, parametrized, hierarchically-called automatically defined 
functions (ADFs) so that an overall program consists of a main result-producing 
branch and one or more reusable and parameterizable automatically defined functions 
(function-defining branches). In addition, architecture-altering operations (Koza, 
Bennett, Andre, and Keane 1999; Koza, Bennett, Andre, Keane, and Brave 1999) 
enable genetic programming to automatically determine the number of automatically 
defined functions, the number of arguments that each possesses, and the nature of the 
hierarchical references, if any, among such automatically defined functions. 

Genetic programming often creates novel designs because it is a probabilistic 
process that is not encumbered by the preconceptions that often channel human 
thinking down familiar paths. For example, genetic programming is capable of 
synthesizing the design of both the topology and sizing for a wide variety of analog 
electrical circuits from a high-level statement of the circuit’s desired behavior and 
characteristics (Koza, Bennett, Andre, and Keane 1999; Koza, Bennett, Andre, Keane, 
and Brave 1999). Five of the evolved analog circuits in that book infringe on 
previously issued patents while five others deliver the same functionality as previously 
patented inventions in a novel way. 

Additional information on current research in genetic programming can be found 
in Banzhaf, Nordin, Keller, and Francone 1998; Langdon 1998; Ryan 1999; Kinnear 
1994; Angeline and Kinnear 1996; Spector, Langdon, O Reilly, and Angeline 1999; 
Koza, Goldberg, Fogel, and Riolo 1996; Koza, Deb, Dorigo, Fogel, Garzon, Iba, and 
Riolo 1997; Koza, Banzhaf, Chellapilla, Deb, Dorigo, Fogel, Garzon, Goldberg, Iba, 
and Riolo 1998; Banzhaf, Poll, Schoenauer, and Fogarty 1998; Banzhaf, Daida, Eiben, 
Garzon, Honavar, Jakiela, and Smith 1999; Poll, Nordin, Langdon, and Fogarty 1999; 
at web sites such as www.genetic-programming.org; and in the Genetic 
Programming and Evolvable Machines journal (from Kluwer Academic Publishers). 



4 Genetic Programming and Control 

Controllers can be represented by block diagrams in which the blocks represent signal 
processing functions, in which external points represent the controller’s input(s) and 
output(s), and in which cycles in the block diagram correspond to internal feedback 
inside the controller. Genetic programming can be extended to the problem of creating 
both the topology and parameter values for a controller by establishing a mapping 
between the program trees used in genetic programming and the block diagrams 
germane to controllers. 
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The number of result-producing branches in the to-be-evolved controller equals the 
number of control variables that are to be passed from the controller to the plant. Each 
result-producing branch is a composition of the functions and terminals from a 
repertoire (below) of functions and terminals. 

Program trees in the population during the initial random generation (generation 0) 
consist only of result-producing branch(es). Automatically defined functions are 
introduced incrementally (and sparingly) into the population on subsequent 
generations by means of the architecture-altering operations. Each automatically 
defined function is a composition of the functions and terminals appropriate for 
control problems, references to existing automatically defined functions, and 
(possibly) dummy variables (formal parameters) that permit parameterization of the 
automatically defined function. Automatically defined functions provide a mechanism 
for internal feedback (recursion) within the to-be-evolved controller. Automatically 
defined functions also provide a mechanism for reusing useful substructures. 

Each branch of each program tree in the initial random population is created in 
accordance with a constrained syntactic structure. Each genetic operation executed by 
genetic programming (crossover, mutation, reproduction, or architecture-altering 
operation) produces offspring that comply with the constrained syntactic structure. 

Genetic programming has recently been used to create a controller for a particular 
two-lag plant and a three-lag plant (Koza, Keane, Yu, Bennett, and Mydlowec 2000). 
Both of these genetically evolved controllers outperformed the controllers designed by 
experts in the field of control using the criteria originally specified by the experts. 

5 Preparatory Steps 

Six major preparatory steps are required before applying genetic programming: (1) 
determine the architecture of the program trees, (2) identify the terminals, (3) identify 
the functions, (4) define the fitness measure, (5) choose control parameters for the run, 
and (6) choose the termination criterion and method of result designation. 

5.1 Program Architecture 

Since there is one result-producing branch in the program tree for each output from 
the controller and this problem involves a one-output controller, each program tree has 
one result-producing branch. Each program tree in the initial random generation 
(generation 0) has no automatically defined functions. However, in subsequent 
generations, architecture-altering operations may insert and delete automatically 
defined functions (up to a maximum of five per program tree). 

5.2 Terminal Set 

A constrained syntactic structure permits only a single perturbable numerical value to 
appear as the argument for establishing each numerical parameter value for each 
signal processing block requiring a parameter value. These numerical values initially 
range from -5.0 to h- 5.0. These numerical values are perturbed during the run by a 
Gaussian mutation operation that operates only on numerical values. Numerical 
constants are later interpreted on a logarithmic scale so that they represent values in a 
range of 10 orders of magnitude (Koza, Bennett, Andre, and Keane 1999). 

The remaining terminals are time-domain signals. The terminal set, T, for the 
result-producing branch and any automatically defined functions (except for the 
perturbable numerical values mentioned above) is 
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T = {REFERENCE_SIGNAL, CONTROLLER_OUTPUT, PLANT_OUTPUT, 
CONSTANT_0}. 

Space does not permit a detailed description of the various terminals used herein 
(although the meaning of the above terminals should be clear from their names). See 
Koza, Keane, Yu, Bennett, and Mydlowec 2000 for details. 

5.3 Function Set 

The functions are signal processing functions that operate on time-domain signals (the 
terminals in T). The function set, F, for the result-producing branch and any 
automatically defined functions is 
F = {GAIN, INVERTER, LEAD, LAG, LAG2, 

DIFFERENTIAL_INPUT_INTEGRATOR, DIFFERENTIATOR, 
ADD_SIGNAL, SUB_SIGNAL, ADD_3_SIGNAL, DELAY, ADFO, ..., 

ADF4}. 

ADFO, ..., ADF4 denote automatically defined functions added during the run by 
architecture-altering operations. 

The functionality of each of the above signal processing functions is suggested by 
their names and is described in detail in Koza, Keane, Yu, Bennett, and Mydlowec 
2000. 

5.4 Fitness 

Genetic programming is a probabilistic algorithm that searches the space of 
compositions of the available functions and terminals. The search is guided by a 
fitness measure. The fitness measure is a mathematical implementation of the high- 
level requirements of the problem. The fitness measure is couched in terms of “what 
needs to be done” — not “how to do it.” 

The fitness measure may incorporate any measurable, observable, or calculable 
behavior or characteristic or combination of behaviors or characteristics. The fitness 
measure for most problems of controller design is multi-objective in the sense that 
there are several different (usually conflicting) requirements for the controller. 

The fitness of each individual is determined by executing the program tree (i.e., 
the result-producing branch and any automatically defined functions that may be 
invoked) to produce an interconnected sequence of signal processing blocks — that is, 
a block diagram for the controller. A SPICE netlist is then constructed from the block 
diagram. The SPICE netlist for the resulting controller is wrapped inside an 
appropriate set of SPICE commands. The controller is then simulated using our 
modified version of the SPICE simulator. The 217,000-line SPICE3 simulator 
(Quarles, Newton, Pederson, and Sangiovanni-Vincentelli 1994) is an industrial- 
strength simulator. It is run as a submodule within our genetic programming system. 
The SPICE simulator returns tabular output and other information from which the 
fitness of the individual is then computed. 

The fitness of a controller is measured using 13 elements consisting of 12 time- 
domain-based elements based on a modified integral of time-weighted absolute error 
(ITAE) and one time-domain-based element measuring disturbance rejection. 

The fitness of an individual controller is the sum (i.e., linear combination) of the 
detrimental contributions of these 13 elements of the fitness measure. The smaller the 
sum, the better. 

The first 12 elements of the fitness measure evaluate how quickly the controller 
causes the plant to reach the reference signal and the controller's success in avoiding 
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overshoot. Two reference signals are used. The first reference signal is a step 
function that rises from 0 to 1 volts at f = 100 milliseconds while the second rises from 

0 to 1 microvolts at t = 100 milliseconds. The two step functions are used to deal with 
the non-linearity caused by the limiter. Two values of the time constant, T, are used 
(namely 0.5 and 1.0). Three values of K are used, namely 0.9, 1.0, and 1.1. Exposing 
genetic programming to different combinations of values of step size, K, and T 
produces a robust controllers and also prevents genetic programming from engaging in 
pole elimination. For each of these 12 fitness cases, a transient analysis is performed 
in the time domain using the SPICE simulator. Table 1 shows the elements of the 
fitness measure in its left-most four columns. 

The contribution to fitness for each of these 12 elements of the fitness measure is 
based on the integral of time-weighted absolute error (ITAE) 

36 

\{t-5)\e{t)\A{e{t))BCdt . 

t=5 

Because of the built-in five-second time delay, the integration runs from time t = 5 
seconds to t = 36 seconds. Here e(f) is the difference (error) at time t between the 
delayed plant output and the reference signal. The integral of time-weighted absolute 
error penalizes differences that occur later more heavily than differences that occur 
earlier. 

We modified the integral of time-weighted absolute error in four ways. First, we 
used a discrete approximation to the integral by considering 120 300-millisecond time 
steps between t = 5 to f = 36 seconds. Second, we multiplied each fitness case by the 
reciprocal of the amplitude of the reference signals so that both reference signals (1 
microvolt and 1 volt) are equally influential. Specifically, B is a factor that is used to 
normalize the contributions associated with the two step functions. B multiplies the 
difference e(t) associated with the 1-volt step function by 1 and multiplies the 
difference e(t) associated with the 1-microvolt step function by 10^ Third, the integral 
contains an additional weight. A, that varies with e(t). The function A weights all 
variation up to 102% of the reference signal by a factor of 1.0, and heavily penalizes 
overshoots over 2% by a factor 10.0. Fourth, the integral contains a special weight, C, 
which is 5.0 for the two fitness cases for which K = I and T = 1, and 1.0 otherwise. 

The 13th element of the fitness measure is based on disturbance rejection. The 
penalty is computed based on a time-domain analysis for 36.0 seconds. In this 
analysis, the reference signal is held at a value of 0. A disturbance signal consisting of 
a unit step is added to the CONTROLLER_OUTPUT at time t = 0 and the resulting 
disturbed signal is provided as input to the plant. The detrimental contribution to 
fitness is 500/36 times the time required to bring the plant output to within 20 
millivolts of the reference signal of 0 volts (i.e., to reduce the effect to within 2% of 
the 1-volt disturbance signal) assuming that the plant settles to within this range within 
36 seconds. If the plant does not settle to within this range within 36 seconds, the 
detrimental contribution to fitness is 500 plus the absolute value of the plant output in 
volts times 500. For example, if the effect of the disturbance was never reduced below 

1 volts, the detrimental contribution to fitness would be 1000. 

A controller that cannot be simulated by SPICE is assigned a high penalty value of 

fitness (10^). 
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5.5 Control Parameters 

The population size, M, was 500,000. A maximum size of 150 points (functions and 
terminals) was established for each result-producing branch and a maximum size of 
100 points was established for each automatically defined function. The other 
parameters for controlling the runs are the default values that we apply to a broad 
range of problems (Koza, Bennett, Andre, and Keane 1999). 

5.6 Termination 

The run was manually monitored and manually terminated when the fitness of many 
successive best-of-generation individuals appeared to have reached a plateau. The 
single best-so-far individual is harvested and designated as the result of the run. 

5.7 Parallel Implementation 

This problem was run on a home-built Beowulf-style (Sterling, Salmon, Becker, and 
Savarese 1999; Bennett, Koza, Shipman, and Stiffelman 1999) parallel cluster 
computer system consisting of 1,000 350 MHz Pentium II processors (each 
accompanied by 64 megabytes of RAM). The system has a 350 MHz Pentium II 
computer as host. The processing nodes are connected with a 100 megabit-per-second 
Ethernet. The processing nodes and the host use the Linux operating system. The 
distributed genetic algorithm with unsynchronized generations and semi-isolated 
subpopulations was used with a subpopulation size of Q = 500 at each of D = 1,000 
demes. Two processors are housed in each of the 500 physical boxes of the system. As 
each processor (asynchronously) completes a generation, four boatloads of emigrants 
from each subpopulation (selected probabilistically based on fitness) are dispatched to 
each of the four toroidally adjacent processors. The migration rate is 2% (but 10% if 
the toroidally adjacent node is in the same physical box). 

6 Results 

The best individual in generation 0 has a fitness of 1926.498. 

The best-of-run controller emerged in generation 129 (figure 1). This best-of-run 
controller has a fitness of 522.605. The result-producing branch of this best-of-run 
individual has 119 points (functions and terminals) and 95, 93, and 70 points, 
respectively, in its three automatically defined functions. Note that genetic 
programming employed a 4.8 second delay (comparable to the five-second plant 
delay) in the transfer function of the evolved pre-filter. This best-of-run controller 
from generation 129 has a better value of fitness for a step size of 1 volt, an internal 
gain, K, of 1.0, and a time-constant, x,of 1.0 (the specific case considered by Astrom 
and Hagglund 1995). 

Figure 2 compares the time-domain response to step input of the best-of-run 
controller from generation 129 (triangles) with the controller in Astrom and Hagglund 
1995 (squares) for a step size of 1 volt, an internal gain, K, of 1.0, and a time-constant, 
T,of 1.0. 

Figure 3 compares the disturbance rejection of the best-of-run controller from 
generation 129 (triangles) with the controller in Astrom and Hagglund 1995 (squares) 
for a step size of 1 volt, an internal gain, K, of 1 .0, and a time-constant, T,of 1 .0. 
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Figure 1 Best-of-run controller from generation 129 for three-lag plant with five-second delay. 



Table 1 compares the fitness of the best-of-run controller from generation 129 and 
the Astrom and Hagglund 1995. Two of the entries are divided by the special weight 
C = 5.0. All 12 entries are better for the genetically evolved controller than for the 
Astrom and Hagglund 1995 controller. 



Table 1 Fitness of two controllers for three-lag plant with five-second delay. 



Element 


Step 


Plant 


Time 


Best-of-run 


Astrom and 




size 


internal 


constan 


generation 


Hagglund 




(volts) 


Gain, K 


t, T 


129 


controller 


0 


1 


0.9 


1.0 


13.7 


27.4 


1 


1 


0.9 


0.5 


25.6 


38.2 


2 


1 


1.0 


1.0 


34.0/5 = 6.8 


22.9 


3 


1 


1.0 


0.5 


18.6 


29.3 


4 


1 


1.1 


1.0 


4.4 


25.4 


5 


1 


1.1 


0.5 


16.3 


22.7 


6 


Ixr® 


0.9 


1.0 


13.2 


27.4 


7 


Ixr® 


0.9 


0.5 


25.5 


38.2 


8 


Ixr® 


1.0 


1.0 


30.7/5 = 6.1 


22.9 


9 


Irr® 


1.0 


0.5 


18.5 


29.3 


10 


Itf® 


1.1 


1.0 


4.3 


25.4 


11 


Itf® 


1.1 


0.5 


16.2 


22.7 


Disturbance 


1 


1 


1 


302 


373 
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Figure 3 Comparison for disturbance rejection. 
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Abstract. This paper proposes a multi-objective evolutionary automated 
design methodology for multivariable QFT control systems. Unlike 
existing manual or convex optimisation based QFT design approaches, 
the ’intelligent’ evolutionary technique is capable of automatically 
evolving both the nominal controller and pre-filter simultaneously to 
meet all performance requirements in QFT, without going through the 
conservative and sequential design stages for each of the multivariable 
sub-systems. In addition, it avoids the need of manual QFT bound 
computation and trial-and-error loop-shaping design procedures, which 
is particularly useful for unstable or non-minimum phase plants for 
which stabilising controllers maybe difficult to be synthesised. 
Effectiveness of the proposed QFT design methodology is validated 
upon a benchmark multivariable system, which offers a set of low-order 
Pareto optimal controllers that satisfy all the required closed-loop 
performances under practical constraints. 



1 Introduction 

Quantitative Feedback Theory (QFT) is well-known as an efficient frequency domain 
controller design methodology that utilises Nichols chart to achieve a desired robust 
design over specified ranges of structured plant parameter uncertainties with and 
without control effector failures [1-3]. The basic idea of QFT is to convert design 
specification on closed-loop response and plant uncertainty into robust stability and 
performance bounds on open-loop transmission of the nominal system as shown in 
Fig. 1 . A fixed structure controller G(s) and pre-filter F(s) is then synthesized using 
gain-phase loop-shaping technique so that the two-degree-freedom output feedback 
system is controlled within specification for any member of the plant templates. For 
multi-input multi-output (MIMO) systems, conventional QFT method requires the 
design process to be turned into a sequence of multi-input single-output (MISO) 
problems before any QFT design procedure can be performed [1-3]. Giving a 
combined solution of controller G(s) = diag[gi(s)] and pre-filter F(s) = [/],], \/ i,j = 1, 
2, ..., m for an m input and m output control problem, the solution of the first set of 
MISO problem is the first transfer function of the diagonal controller gi(^) and/n(s). 



S. Cagnoni et al. (Eds.): EvoWorkshops 2000, LNCS 1803, pp. 178-194, 2000. 
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Then a new set of MISO problem is defined based on the plant, gi(s),/n(j),/i 2 (s),..., 
/im(s) which results to the latter solution of the controller g 2 (s) and pre-filter / 2 i(s), 
fiiis), ■■•,/ 2 m(s) and etc., Hence, this design method leads to an overall design of n? 
MISO equivalent loops and its size grows exponentially to the m number of 
inputs/outputs, which can be very tedious to the designer [3]. 



R{s) 



F(s) i G(s) 



-M Pi^) 






Fig. 1. A typical output feedback multivariable control system in QFT 



As pointed out in [4], there are no other design methods than the manual trial- 
and-error process could be employed to determine these series of loops. Moreover, 
this sequential MISO design procedure may be conservative since the solution of a set 
of MISO is highly dependent and restricted to the former sets of MISO solution. The 
user may thus have to repeat or re-restart the design procedure from the first MISO 
loop synthesis, if there exist any set of MISO solutions that is unfeasible due to 
improper or overly design of the previous sets of MISO loops. Besides, QFT bounds 
in Nichols chart for each specification of all frequency points must be acquired before 
the design, which is often an exhaustive trial-and-error process. The reason is that, for 
every frequency point with sufficiently small frequency interval, the template needs to 
be manually shifted up or down on the Nichols chart until the gain variation of the 
template is equal to the gain variation allowed for any particular robust specification 
at that frequency. In addition, only the controller can be synthesized via QFT bound 
computation using the conventional loop-shaping method. Another independent 
design task has to be accomplished in order to obtain the pre-filter within a two-stage 
design framework for each set of MISO solution. 

The acquisition of an optimal QFT controller is in fact a multi-objective multi- 
modal design optimisation problem that involves simultaneously determining multiple 
controller and pre-filter parameters to satisfy different competing performance 
requirements, such as sensitivity bounds, cross-coupling bounds, robust margin and 
etc.. To solve these problems, a few analytical/mathematics oriented optimisation or 
'automatic design' techniques have recently been investigated and developed [5-7]. 
These convex based optimisation approaches, however, impose many unpractical or 
unrealistic assumptions that often lead to very conservative designs or are hard in 
finding the global Pareto optimal solutions in the multi-objective multi-dimensional 
design space. With the view of tackling these drawbacks and automating the QFT 
design procedure, computerised 'intelligent' trial-and-error based methodology based 
on evolutionary optimisation has been proposed and successfully applied to industrial 
or benchmark applications [8-10]. 

This paper further develops the multi-objective 'intelligent' automated QFT 
design methodology to MIMO control system using a high performance evolutionary 
algorithm toolbox [11]. Unlike existing methods, the evolutionary QFT design 
approach is capable of concurrently evolving the controller and pre-filter for the entire 
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set of MISO sub-systems to meet all performance requirements in QFT, without going 
through the conservative and sequential design stages for each of the MISO sub- 
systems. Besides, the evolutionary toolbox is built with comprehensive user interface 
and powerful graphical displays for easy assessment of various simulation results or 
trade-offs among the different design specifications. The paper is organized as 
follows: The various QFT design specifications and the role of the MOEA toolbox in 
the multivariable QFT design are given in Section 2. Validation of the proposed 
methodology against a benchmark MIMO system is illustrated in Section 3. 
Conclusions are drawn in Section 4. 

2. Evolutionary Automated Multivariable QFT Design 



2.1 Multi-Objective QFT Design Specifications 

As mentioned in the Introduction, there are a number of usually conflicting design 
objectives need to be satisfied concurrently in multivariable QFT designs. In contrast 
to the conventional two-stage loop-shaping approach, these performance requirements 
are formulated as a multi-objective design optimisation problem here. The aim is thus 
to concurrently design the nominal controller G(s) and pre-filter F(s) in order to 
satisfy all the required specifications as described below: 



(i) Stability (RHSP) 

The cost of stability, RHSP, is included to ensure stability of the closed-loop system, 
which could be evaluated by solving the roots of the characteristic polynomial. 
Clearly, a stable closed-loop system for all the plant templates p requires a zero 
value of RHSP, 



RHSP='^Nr\ 



real] 



pole\ 



P,GF 

I + PG 



• > 0 , yp^e p 



( 1 ) 






In order to ensure internal stability and to guarantee no unstable pole and non- 
minimum phase zero cancellations, it is desired that a minimum phase and stable 
controller be designed. This implies that the search range for a polynomial coefficient 
set is limited to either the first or the third ’quadrant’, i.e., all coefficients in the 
numerator or denominator must be of the same sign [10]. The poles and zeros of the 
controllers can be calculated explicitly to avoid RHP cancellations or alternatively, 
the Horowitz method for QFT design of unstable and non-minimum phase plants can 
be used, i.e., QFT bounds for an unstable/non-minimum phase nominal plant can be 
translated to those for a stable and minimum phase plant, if necessary. 



(ii) Robust Upper and Lower Tracking Performance (ERRUT & ERRLT) 

The cost of upper tracking performance of the i* diagonal element of MIMO closed- 
loop transfer function, given by ERRUfij), is included to address the specification of 
upper tracking bound as shown in Fig. 2. It is computed as the sum of absolute error 
at each frequency point as given by, 

n 

ERRUf,., = 

k=l 



( 2 ) 
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where n is the total number of interested frequency points; is the difference 

between the upper bound of the (/,/) element of the closed-loop transfer function 
CL(i i)u and the pre-specified upper tracking bound at frequency if the upper 
bound of the closed-loop system is greater than the pre-specified upper tracking 
bound or less than the pre-specified lower tracking bound !(; ,)/,; otherwise, is 

equal to zero as illustrated in Fig. 2, for which the length of the vertical dotted lines at 
each frequency represents the magnitude of e(, 

dB 




Fig. 2. Computation of upper tracking performance for the i* diagonal element 

The cost for lower tracking performance of the diagonal element of the 
closed-loop transfer function, given by ERRLT^iiy, can be defined as the sum of 
absolute error at each frequency point, 

n 

ERRLT,,^='^\e,,^,,M)\ ( 3 ) 

k=l 

where n is the number of frequency points; is the difference between the lower 
bound of the closed-loop system CL(,,,)i and the pre-specified lower tracking bound 
T(i,i)L, if the lower bound of the closed-loop system is greater than the pre-specified 
upper tracking bound T(, ,)(/ or less than the pre-specified lower tracking bound 
Otherwise, is equal to zero as illustrated in Fig. 3. 



dB 




Fig. 3. Computation of lower tracking performance for the i* diagonal element 
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(Hi) Cross-coupling Performance (ERRUC) 

Apart from addressing the tracking performance of diagonal elements, it is also 
essential to reduce the coupling effect of the off-diagonal transfer functions for all the 
plant templates in MIMO QFT control system design. Since the objective is to reduce 
the gain and bandwidth of the off-diagonal transfer function, only the upper bounds in 
the frequency response need to be prescribed [4]. The upper bound of coupling effect 
represented by transfer function for the off-diagonal transfer functions (ij) 

where i ^ j can be defined according to the allowable gain K and the bandwidth 
between vvi and W 2 , which is shown in Fig. 4 and takes the form of 



T'u.jW - / 



K{—s) 

w, 



(4) 



-j-l-1 



i-(-l 



The cost of cross-coupling effect for the off-diagonal elements of MIMO closed-loop 
system, given by ERRUC^ij), is include to address the specification of upper cross- 
coupling bound and is computed as the sum of absolute error at each frequency point, 



ERRUC,^^, = X|%;>cK)| > for (5) 

*=i 



where n is the number of frequency points; e^ij^ud^k) at (»*, is the difference between 
the upper bound of the closed-loop system CL^ij^u and the pre-specified upper cross- 
coupling bound if the upper bound of the closed-loop system is greater than the 
pre-specified upper cross-coupling bound as illustrated in Fig. 4; Otherwise, 

is equal to zero as given by, 






1 0 



Otherwise 



for i ^ j (6) 




Fig. 4. Computation of upper cross-coupling performance for off-diagonal elements 
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(iv) Robust Margin (RM) 

Practical control applications often involve neglected uncertainties or unmodelled 
dynamics at the high frequency. Because of these missing dynamics, the nominal 
model used in control system design is often inaccurate either through deliberate 
neglect or due to the lack of understanding of the physical process [12]. In order to 
address these unmodelled uncertainties, the uncertain feedback system with inverse 
multiplicative plant uncertainty, P^^{s) = P(s){l +W,, as shown in Fig. 5 is 



consdired in the QFT design. The robust margin specification that addresses the 
closed-loop stability due to the inverse multiplicative plant uncertainty for an 
uncertainty weighting function Wn can be defined as [12], 



RM 



IM 



1 

I +mjco) 



1 

^W„Uco ) ’ 



\/a> 



(7) 



where L.(jco) is the i* open-loop transfer function with the y* loop being closed in an 



MIMO system, which is simply the loop transmission P(jco)G{jco) in an SISO 



system. 




Fig. 5. Feedback system with inverse multiplicative uncertainty 



(v) Sensitivity Rejection (RS) 

The robust sensitivity rejection is to find a QFT controller that minimises the 
maximum amplitude of the regulated output over all possible disturbances of bounded 
magnitude. A general structure to represent the disturbance rejection is given in Fig. 
6, which depicts the particular case where the disturbance enters the system at the 
plant output. The mathematical representation is given by, 

5 = ^ = {/ + PQ)GQ)r (8) 

The matrix S(s) is known as the disturbance rejection. The maximum singular values 
of S determines the disturbance attenuation since S is in fact the closed-loop transfer 
from disturbance D to the plant output Y. 




Fig. 6. Formulation of a sensitivity rejection problem 
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The disturbance attenuation specification for the closed-loop system may thus be 
written as, 

ct(5)<||w/|L ^||W,5|L<1 (9) 

where a defines the largest singular value and the desired disturbance 
attenuation factor, which is a function of frequency to allow a different attenuation 
factor at each frequency. 



(vi) High Frequency Gain Roll-ojf (HFG) 

The high frequency gain performance, HFG, is included to reduce the gain of loop 
transmission L(s) at the high frequency in order to avoid the high-frequency sensor 
noise and the unmodelled high-frequency dynamics/harmonics that may result in 
actuator saturation and instability. The high frequency gain of loop transmission L(s) 
is given as, 

lims^Lis) (10) 



where r is the relative order of L(s). Since only the controller in the loop transmission 
is to be optimised, this performance requirement is equivalent to the minimization of 
high frequency gain of the controller or the magnitude of bja^ for a controller 
structure given as [9], 



G„,oW = 



h h ^ ‘ h 






m—l , 

S H Gr; 



(i,i)0 



( 11 ) 



where n and m is the order of the numerator and denominator for the (/,/) element of 
diagonal controller G(s), respectively. 



2.2 Evolutionary Algorithm Toolbox and Its Role in QFT Design 

Although the multi-objective optimisation based QFT design method has the merit of 
avoiding conventional independent two-stage controller synthesis or the tedious 
sequential designs to determine the series of loops for each of the MISO sub-system, 
the approach needs to search for multiple optimised controller and pre-filter 
coefficients to satisfy a set of non-commensurable and often competing design 
specifications. Such an optimisation problem is often semi-infinite and generally not 
everywhere differentiable [9]. It is thus hard to be solved via traditional numerical 
approaches that often rely on a differentiable performance index, which forms the 
major obstacle for the development of a generalised numerical optimisation package 
for QFT control applications. 

This paper proposes an evolutionary automated design methodology for the 
multi-objective QFT control optimisation problem. Fig. 7 shows a general 
architecture for the computer aided control system design (CACSD) automation of 
MIMO QFT control system using a multi-objective evolutionary algorithm (MOEA) 
toolbox. The design cycle accommodates three different modules: the interactive 
human decision-making module (control engineer), the optimisation module (MOEA 
toolbox) and the QET control module (system and specifications). According to the 
system performance requirements and a-priori knowledge on the problem on-hand if 
any, control engineers may specify or select the desired QFT specifications as 




Automatic Design of Multivariable QFT Control System 



185 



discussed in previous sections to form a multi-objective function, which need not 
necessary he convex or differentiable. Based on these design specifications, responses 
of the control system consists of the set of input/output signals, the plant template as 
well as the candidate controller G(s) and pre-filter F(s) recommended from the 
optimisation module are simulated as to determine the different cost values for each 
design specification in the multi-ohjective function. 

According to the evaluation results of the multi-objective function in the control 
module and the design guidance such as goal or priority information from the 
decision-making module, the optimisation module (MOEA toolbox) automates the 
QFT design process and intelligently searches for the ’optimal’ controller and pre- 
filter parameters simultaneously that best satisfy the set of QFT performance 
specifications. On-line optimisation progress and simulation results, such as the 
design trade-offs or convergence are displayed graphically and feedback to the 
decision-making module. In this way, the overall QFT design environment is 
supervised and monitored effectively, which helps control engineers to make 
appropriate actions such as examining the competing design trade-offs, altering the 
design specifications, adjusting goal settings that are too stringent or generous, or 
even modifying the QFT control and system structure if necessary. This man-machine 
interactive design and optimisation process maybe proceeded until the control 
engineer is satisfied with the required performances or after the design specifications 
have been met. Such an evolutionary automated approach allows the QFT design 
problem as well as the interaction with optimisation process to be closely linked to the 
environment of that particular application. Control engineer, for most of the part, is 
not required to deal with any details that are related to the optimisation algorithm or to 
go through the manual trial-and-error two-stage and sequential design as adopted in 
conventional QFT design methods. 

The MOEA toolbox [11] has been developed under the Matlab [13] 
programming environment, which is effective for global optimisation and assessment 
of multi-objective design trade-off scenarios, aiding at decision-making for an optimal 
solution that best meets all design specifications. It is also capable of handling 
problems with constraints and incorporating advanced goal and priority information 
with logical AND/OR operations for higher-decision support. Besides, it is fully 
functioned with graphical user interface (GUI) and is ready for immediate use with 
minimal knowledge on evolutionary computing or Matlab programming. The toolbox 
also allows the different representation of simulation results in various formats, such 
as text files or graphical displays for the purpose of on-line viewing and analysis. 
With the toolbox, designer merely needs to give a model file relating to his/her 
particular optimisation problem, and configures the problem based on a few simple 
GUI setups. Further descriptions of the toolbox and GUIs maybe referred to [11] or 
the tutorials in the toolbox, which is freely available for downloading at 
http://web.singnet.com.sg/~kavchen/moea.htm . 
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Fig. 7. A general evolutionary design automated QFT control framework 

3. A Benchmark MIMO QFT Design Problem 



The benchmark MIMO QFT control problem given in [14] is studied in this section, 
which is shown in Fig. 1 with the MIMO uncertain plant sets given as, 

a 3 -I- 0.5a 1 






A(^) 

I 



A(^) 

8 

A(^) J 



( 12 ) 



where A(s) = -l-0.03as-i-10 and a e [6, 8] . Apart from the few design speciciations 

studied by [14], additional performance requirements such as robust tracking and 
cross-coupling specifications are included here for wider consideration of the QFT 
design objectives, which subsequently adds to the design difficulty and complexity. 
The specification of high frequency gain [9, 10] is also incorporated to avoid any 
high-frequency sensor noise and unmodelled high-frequency dynamics/harmonics. 
The various closed-loop performance requirements for this MIMO QFT design are 
formulated as follows: 



(i) Robust Tracking Bounds for diagonal transfer functions: 

^ {(O ) , for (=1,2 



Upper Tracking Model: 



1.9 xlo'^Qajj-r 6.4x10^ 



'( 2 , 2 )(/ 



(®) = 



{jmf + 2.3 X 10^ + 1 .9 X 10‘*(» -r 6.4 x 10^ 

6.4 xl0\yVu)-H 3.4x10^ 



(jo)f +I.5xl0^{jcaf -H8xlO^Q0)-r3.4xlO^ 



Lower Tracking Model: 



(13) 

(14a) 

(14b) 
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^(2,2)l(®) 



1x10® 

ijcof +3x10^0)2 +3xl0'^(» + 1x10® 
2.5x10® 

{jcof + 2.3 X 10^0)^ + 1.5 X lO'^O) + 2.5 x 10® 



(ii) Robust Cross-Coupling Bounds for off-diagonal transfer functions: 
\chij) (»| ^ (®) ,for i and i, j= 1,2 

where, 

^(l,2)t/ (®) = 

^(2,1)C/ (®) = 



0.0032 (;w) 

[0.016(;®) + l] [0.016(;®) + l] 

6.3xl0~®(jw) 

[0.016(;®) + l] [0.016(» + l] 



(15a) 

(15b) 

(16) 

(17a) 

(17b) 



(iii) Robust Sensitivity Rejections for full matrix transfer functions: 

|S; j(y®)| < aij(jco) , for m < 10 (18) 

where, a,y = 0.01 0), for i = j; stij = 0.005 ty, for i ^ j 



(iv) Robust Stability Margin: 



1 + L,_,(;®) 



,for V i = 1,2 , and co > 0 



(19) 



The performance bounds of QFT are computed within a wide frequency range of 
10’^ rad/s to 10® rad/s. Without loss of generality, the structure of the diagonal 
controller G(s) is chosen in the form of a general transfer function [9] as given by, 

G,„(^) = ^ , V a„ e 9T, for / = 1, 2 (20) 

n=0 

Note that the controller can also be designed by refining position of poles and zeros 
directly or by using other structures such as the realisable (non-ideal) PID structure if 
desired. The filter is fixed to a full matrix first-order transfer function as it is relevant 
to the tracking and cross-coupling bound in the frequency response. Since the 
resultant pre-filter must satisfy lim[F(^)] = l for a step forcing function [9], the 



structure of pre-filter F(s) is chosen as a full matrix first-order transfer function as 
given by. 



f.jW-- 






C„s 



V c,, e 91^ , for V i,j = 1,2 



(21) 
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Apart from most default settings, the evolutionary toolbox has been configured 
with a population and generation size of 200 and 100, respectively. To guide the 
evolutionary QFT design optimisation process, goal and priority for each of the 
performance requirements maybe included optionally as shown in Fig. 8. Although 
determination of the goal and priority maybe a subjective matter and depends on the 
performance requirements, it maybe unnecessary and can be ignored for a ‘minimum- 
commitment’ design [9]. In principle, any number or combination of QFT 
performance specifications can be added to the design using the multi-objective 
evolutionary optimisation approach if necessary. 




Fig. 8. Settings of the MOEA toolbox for the benchmark QFT design problem 

A powerful feature of the evolutionary QFT design is that it allows on-line 
examination of different trade-offs among the multiple conflicting specifications, 
modification of existing objectives and constraints, and zoom into any region of 
interest before selecting one final set of controller and pre-filter for real time 
implementation. The trade-off graph of the resultant QFT control system is shown in 
Fig. 9, where each line representing a solution found by the evolutionary optimisation. 
The cost of objectives such as stability (RHSP), robust tracking and cross coupling 
performances {ERRUT and ERRLT) are labelled as objectives 1-7, which are all equal 
to zero as desired according to the goal settings in Fig. 8. The x-axis shows the design 
specifications, the y-axis shows the normalised cost for each objective and the cross- 
mark shows the desired goal setting for each performance requirement. Clearly, trade- 
offs between adjacent specifications results in the crossing of the lines between them, 
whereas concurrent lines that do not across each other indicating the specifications do 
not compete with one another. For example, the robust sensitivity objective of 12 
(RS21) and 13 (RS22) are not competing with each other, whereas the robust margin 
objective 8 (RMl) and 9 (RM2) appear to compete heavily, as expected. The 
information contained in this trade-off graph also suggests that lower goal settings for 
robust sensitivity (objectives 10-13) are possible, which can be further optimised to 
arrive at an even better robust performance. 
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Fig. 9. Trade-off graph of the evolutionary designed QFT control system 

Note that the evolutionary QFT design also allows engineers to divert the 
evolution to other focused trade-off region or to modify any preferences on the 
current specification settings after observation for a number of generations. For 
example, the designer can change his preference and decide to reduce the 9* goal 
setting for robust margin (RM2) from 1.8 to 1.3. Fig. 10 illustrates the behaviour of 
the evolution upon the modification of this goal setting after the evolutionary QFT 
design in Fig. 9. Due to the sudden change of a tighter goal setting, initially none of 
the individuals manage to meet all the required specifications as shown in Fig. 10(a). 
After continuing the evolution for 2 generations, the population moves towards 
satisfying the objective of RM2 as shown in Fig. 10(b) at the performance expense of 
other objectives since they are highly correlated and competing to each other. The 
evolution continues and again leads to the satisfaction of all the required goal settings 
including the stricter setting of objective RM2 as shown in Fig. 10(c). Clearly, this 
man-machine interactive design approach has enabled QFT designers to divert the 
evolution into any interested trade-off regions or to modify certain specifications and 
preferences on-line, without the need of restarting the entire design process as 
required by conventional QFT design methods. 




(a) On-line goal modification of robust margin objective (RM2) 



Normalized costs 
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2 4 6 8 10 12 14 



Objectives 

(b) After 2 generations 




2 4 6 8 10 12 14 



Objectives 

(c) After another 2 generations 



Fig. 10. Effects of the evolution upon the on-line modification of goal setting 



Fig. 1 1 shows the robust tracking performances in the frequency domain for the 
two diagonal elements of the closed-loop system. It can be seen that all the frequency 
responses of CLu and CL^ for both the diagonal channels are located successfully 
within their respective pre-specified tracking hounds of Tu and T^. Besides, the 
coupling effect from the off-diagonal elements of the closed-loop system for all the 
plant templates has also been reduced satisfactory and successfully bounded by the 
upper coupling hound with minimal gain and bandwidth of the off-diagonal transfer 
functions as shown in Fig. 12. 





Fig. 11. The tracking performance in the frequency domain 
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Frequency (rad/sec) 

(a) Off-diagonal element of (1,2) 




Frequency (rad/sec) 



(b) Off-diagonal element of (2,1) 



Fig. 12. The cross-coupling performance in the frequency domain 



Figs. 13 and 14 show the unit step tracking and coupling performances for all 
the plant templates in the time domain for a random selected set of evolutionary 
designed controller and pre-filter. Clearly, all the time domain tracking and coupling 
performances have been satisfied successfully and within the required prescribed 
tracking bounds, as desired. 




Time (sec) 



(a) Diagonal element of (1,1) 




Time (Sec) 



(b) Diagonal element of (2,2) 



Fig. 13. The tracking responses of the diagonal elements in the closed-loop system 
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Time (Sec) Time (sec) 

(a) Off-diagonal element of (1,2) (b) Off-diagonal element of (2,1) 



Fig. 14. The coupling responses of the off-diagonal elements in the closed-loop 
system 

To illustrate robustness of the evolutionary designed QFT control system on 
disturbance rejection, a unit step disturbance signal was applied to the MIMO system. 
Output disturbance responses for all the final Pareto optimal controllers at each 
element of the closed-loop transfer matrix are illustrated in Fig. 15. Clearly, the unit 
step disturbance has been successfully attenuated to zero eventually for all the 
different values of parameter uncertainties, as quantified by the performance 
specification of robust sensitivity rejection. 




Time (sec) Time (sec) 

(a) Disturbance response of element (1,1) (b) Disturbance response of element (1,2) 
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(c) Disturbance response of element (2, 1) (d) Disturbance response of element (2, 2) 

Fig. 15. Output responses for the unit step disturbance in the time domain 

4 Conclusion 

This paper has analysed difficulties in existing QFT design techniques for 
multivariable control systems. To address these design deficiencies, an automated 
multivariable QFT design methodology using a high performance MOEA toolbox has 
been proposed. Unlike existing design methods, the ’intelligent’ design approach is 
capable of automatically evolving both nominal controller and pre-filter 
simultaneously to meet all performance requirements in QFT, without going through 
the sequential design stages for each of the multivariable sub-systems. Besides, the 
approach also avoids the need of manual QFT bound computation and trial-and-error 
loop-shaping procedures as required by conventional means. It is shown that control 
engineers’ expertises as well as goal and priority information can be easily included 
and modified on-line according to the evolving trade-offs, instead of repeating or 
restarting the whole design process. It is obvious that the proposed evolutionary QFT 
design framework is fully expandable to on-line design optimisation and 
implementation. This can be realised either via the hard and software systems such as 
dSPACE [15] or MIRCOS [16] for graphical programming and real-time operation to 
provide necessary linkages between the toolbox and the physical environments. Apart 
from the developments for on-line adaptation, the multi-objective evolutionary QET 
design paradigm is currently being extended to robust control of nonlinear systems 
and to incorporate other design specifications such as economical cost consideration. 
Progress and results will be reported in due course. 
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acc rac than that o tain d sing th traditional od Is. h d lop d th r- 
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= -k 




+ ( t - ) 
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= a i nt t p rat r 

= st ad stat oil t p rat r at th otto of th inding ith th 
op rating load 

= oil t p rat r ris a o a i nt at th otto of th inding nd r 

th rat d load 

= th ratio of load loss (at rat d load) to no-load loss 
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load 
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rat d load 
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11 th para t rs in ol d in this od 1, s eh as , and , ar s all d t r- 
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th st ad -stat r lationship t n th , , a i nt t p rat r and 

transfer r load ratio; th s cond cone rns th t p rat r ariation ora 
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th otto -oil and top-oil t p rat r s. h as r d para t rs, pro id d 
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Abstract. In present days, most of the design activity is performed at a high 
level of abstraction, thus designers need to be sure that their designs are 
syntactically and semantically correct before starting the automatic synthesis 
process. The goal of this paper is to propose an automatic input pattern 
generation tool able to assist designers in the generation of a test bench for 
difficult parts of small- or medium- sized digital protocol interfaces. The 
proposed approach exploit a Genetic Algorithm connected to a commercial 
simulator for cultivating a set of input sequence able to execute given 
statements in the interface description. The proposed approach has been 
evaluated on the new ITC 99 benchmark set, a collection of circuits offering a 
wide spectrum of complexity. Experimental results show that some portions of 
the circuits remained uncovered, and the subsequent manual analysis allowed 
identifying design redundancies. 



1 Introduction 

In the past years, the design flow of protocol interfaces, and Application Specific 
Integrated Circuits (ASICs) in general, experienced radical changes. Due to the 
maturity of automatic logic synthesis tools most of the design activity is now 
performed at high level of abstraction, such as register transfer level (RT), instead of 
low level such as gate. The new methodology dramatically increases designer 
productivity since high-level descriptions are more readable and considerably smaller. 

One important step of the new design flow consists of design validation, i.e., the 
verification that the design is syntactically and semantically correct before starting 
automatic logic synthesis. Although many techniques have already been proposed in 
the CAD literature (e.g., static checks, formal verification [HuCh98], mutation testing 
[AHRo98]), none has gained enough popularity to compete with the current industrial 
practice of validation by simulation. Verification engineers resort to extensive 
simulation of each design, and of the complete system, in order to gain confidence 
over its correctness. 

This situation is far from ideal, and designers need to face many difficulties. At the 
present days, simulation technology is effective enough for synthesized circuits. But 
when it comes to mixed-signal circuits, or to circuits containing embedded cores, or to 
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a complete system composed of a network of several interconnected interfaces, 
simulation is unable to provide the needed versatility. 

Even restricting our attention to medium-sized, digital interfaces, the fundamental 
issue of measuring the test bench quality can be considered still unanswered. Many 
metrics have been proposed to evaluate the thoroughness of a given set of input 
stimuli, often adopted from the software testing domain [Beiz90], ranging from 
statement or branch coverage, state coverage (for finite state machine controllers), 
condition coverage (for complex conditionals), to the more complex path coverage. 
Many variants have been developed, mainly to cater for observability [DKGe96] and 
for the inherent parallelism of hardware descriptions [TAZa99], that are not taken into 
account by standard metrics. Since no well established metric is yet widely accepted 
for validation, some authors also propose to measure the quality of validation patterns 
with the stuck-at fault coverage. 

Several products (normally integrated into existing simulation environments) are 
now available that provide the user with the possibility of evaluating the coverage of 
given input stimuli with respect to a selected metric. Designers can therefore pinpoint 
the parts of their design that are poorly tested, and develop new patterns specifically 
addressing them. Currently, this is a very time consuming and difficult task, since all 
the details of the design must be understood for generating suitable input sequences. 
The right trade-off between designer s time and validation accuracy is often difficult 
to find, and this often results in under-verified circuits. Moreover, in the generation of 
test vectors the designer may be biased by his knowledge of the desired system or 
module behavior, so that he often fails in identifying input sequences really able to 
activate possible critical points in the description. 

When faced with this problem, the CAD research community traditionally invested 
in formal verification [GDNe91] [HuCh98], in the hope that circuits can be proven 
correct by mathematical means. Although formal verification tools give good results 
on some domains, they still have too many limitations or they require too much 
expertise to be used as a mainstream validation tool. Designers are left waiting for the 
perfect formal verification system, while few or no innovative tools help them with 
simulation-based validation. 

The goal of this paper is to propose GIP-PI (Genetic Input Pattern generator for a 
Protocol Interface). GIP-PI is an automatic input pattern generation tool able to assist 
designers in the generation of a test bench for difficult parts of small- or medium- 
sized digital protocol interfaces. The proposed approach belongs to a brand new 
framework that can be called approximate validation, which explicitly relinquishes 
exactness in order to gain the ability of dealing with realistic designs. This philosophy 
has already been successfully applied in different areas: validation of the 
implementation of protocol interfaces [CSSq99c]; automatic test pattern generation 
[CSSq99a]; low-level [CSSq98] and mixed-level [CSSq99b] equivalence validation. 
Although the goal of this paper is completely different from the previous applications, 
all these methodologies share a common quality: being able to deal with real circuits 
exploiting an evolutionary algorithm. 

GIP-PI employs a Genetic Algorithm, interacting with a VHDL simulator, for 
deriving an input sequence able to execute a given statement, or branch, in the high- 
level description. Whenever the test bench quality, as measured by one of the 
proposed metrics, is too low, our tool can be used to generate test patterns that are 
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able to stimulate the parts of the design that are responsible for the low metric. The 
designer must manually analyze only those parts of the description that the tool failed 
to cover. Experimental results show that only a small fraction of difficult statements 
remain uncovered, and that many of them, upon closer inspection, indeed contain 
design errors or redundancies. 

While no metric is yet widely accepted by validation teams, we aimed at evaluating 
the effectiveness of our approach using some pre-defmed metric. The algorithm is 
quite easily adapted to different metrics, but for the sake of the experiments we 
adopted branch coverage as a reference. We developed a prototypical system for 
generating test patterns based on branch coverage, applicable to synthesizable VHDL 
descriptions. We aim at addressing moderately sized circuits, that usually can not be 
handled by formal approaches, and at working directly on the VHDL description, 
without requiring any transformation nor imposing syntax limitations. 

The approach has been evaluated on the new ITC 99 benchmark set [ITC99], a 
collection of circuits described in high-level (RT) VHDL that offers a wide spectrum 
of complexity. Manually derived validation suites did not adequately cover all parts of 
the designs, and new sequences have been generated by the tool to increase the 
overall coverage. Experimental results show that some portions of the circuits 
remained uncovered, and the subsequent manual analysis allowed identifying design 
redundancies. 

Section 2 gives an overview over the proposed approach for test bench generation, 
experimental are presented in Section 3 and Section 4 concludes the paper. 



2 RT-level Test Bench Generation 

The goal of test bench generation is to develop a set of input sequences that attain 
the maximum value of a predefined validation metric. Despite this implementation of 
GIP-Pl is tuned for simulating high-level VHDL network interfaces only, the 
proposed method could be easily extended to deal with any simulable descriptions. 
For instance, given a protocol specification in ESTELLE, and with a reduced effort, 
GIP-Pl could eventually generate a set of stimuli (events) to validate the protocol 
description itself 



2.1 Adopted Metric 

Most available tools grade input patterns according to metrics derived from 
software testing [Beiz90]: statement coverage and branch coverage are the most 
widely known, but state/transition coverage (reaching all the states/transitions of a 
controller) and condition coverage (controlling all clauses of complex conditionals) 
are also used in hardware validation. Path coverage, although often advocated as the 
most precise one, is seldom used due to its complexity, and because it loses 
meaningfulness when multiple execution threads run concurrently in parallel 
processes. Some recent work extends those metrics to take also into account 
observability [DGKe96] and the structure of arithmetic units [TAZa99]. Those 
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extensions are essential when the sequences have to be used as test patterns to cover 
stuck-at faults, but for validation they have lower importance since internal values are 
available. 

The metric we adopt in this paper is branch coverage, although the tool can be 
easily adapted to more sophisticated measures. Also, since synthesizable VHDL is a 
structured language, complete statement coverage implies complete branch coverage, 
and the tool takes advantage of this simplification. 



2.2 Overall Approach 

The adopted approach is an evolution of the one presented in [CPSo97], where a 
Genetic Algorithm uses a simulator to measure the effectiveness of the sequences it 
generates. Instead of trying to justify values across behavioral statements, that would 
require solving Boolean and arithmetic constraints [FADe99], thanks to the nature of 
Genetic Algorithms we just need to simulate some sequences and analyze the 
propagation of values. Each sequence is therefore associated with the value returned 
by a fitness function, that measures how much it is able to enhance the value of the 
validation metric, and the Genetic Algorithm evolves and recombines sequences to 
increase their fitness. 

The fitness function needs to be carefully defined, and accurately computed. In 
particular, the fitness function can not be just the value of the validation metric: it 
must also contain some terms that indicate how to increase the covered branches, not 
just to count the already covered ones. In a sense, the fitness function includes a 
dominant term, that measures the accomplished tasks (covered branches), and 
secondary terms, that describe sub-objectives to be met in order to cover new 
branches. 

The computation of such function is accomplished by analyzing the simulation 
trace of the sequence, and by properly weighting the executed assignments, 
statements, and branches according to the target statements. In the implementation, to 
avoid arbitrary limitations in the VHDL syntax, simulation is delegated to a 
commercial simulator that runs an instrumented version of the VHDL code and 
records the simulation trace in the transcript file. Such trace is then interpreted 
according to control- and data-dependencies, that are extracted from a static analysis 
of the design description. Figure 1 shows a simplified view of the overall system 
architecture. 



2.3 VHDL Analysis 

The goal of the algorithm is to achieve complete coverage, but for efficiency 
reasons we do not consider each statement separately, and we group them into basic 
blocks [ASU186]: a basic block is a set of VHDL statements that are guaranteed to be 
executed sequentially, i.e., they reside inside a process and do not contain any 
intermediate entry point nor any control statement (if, case, ). All the operations 
required for code instrumentation, dependency analysis, branch coverage evaluation, 
and fitness function computation are performed at the level of basic blocks. 
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Since the Genetic Algorithm exploits the knowledge about data and control 
dependencies, we need to extract that information from the VHDL code: for this 
reason, we build a database containing a simplified structure and semantics of the 
design. The database is structured as follows: 

• The hierarchy of component instantiations inside different entities is flattened 
(Cl and C2 in the figure 2). A dictionary of signal equivalencies is also built, that 
allow us to uniquely identify signals that span multiple hierarchical levels. 




Figure 1: System architecture 

• All VHDL processes occurring in the flattened circuit are given a unique 
identifier (P; in the figure 2). This operation also converts standalone concurrent 
statements into their equivalent process. The design is thus represented as a 
network of processes interconnected by signals. 




Figure 2: Abstract representation of RT-level designs 

• Each process is analyzed to define its interface, in terms of signals that it reads 
and writes. 

• The sequential part of each process is analyzed, its control flow graph (CFG) is 
extracted, and statements are grouped in basic blocks (BBs). The control 
structure of the process is described as a control flow of basic blocks (figure 2 
reports the CFG for process P2). 
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• A dependency matrix between basic blocks is computed, by assigning a 
probability that a basic block will be executed, given that another block has just 
been executed. These correlation probabilities take into account the branching 
and looping nature of the control flow. 

• Each basic block is entered, and the data flow graph (DFG) of the operations that 
occur inside each basic block is extracted. Since a basic block consists of 
multiple statements and/or conditions, multiple dependencies are associated to a 
single block. Fig. 2 shows the DFG for the basic block BB4 of process P2. 



2.4 Genetic Algorithm 

The Genetic Algorithm (GA) is based on encoding potential test sequences as 
variable length bit matrices. A number of such sequences are randomly generated and 
constitute an initial population', the goal of the GA is to evolve this population to 
increase its fitness value. The fitness function measures the closeness of a sequence to 
the goal. Currently, in GIP-PI the genetic algorithm is run several times, each time for 
a different target. Moreover, the fitness function assumes two different forms: 

• In the initial phase, all basic blocks are considered simultaneously. The goal is to 
generate a set of sequences S that activate most of the blocks, for identifying 
easy-to-execute blocks. The fitness function is simply the number of activated 
blocks over the total number of blocks: 

_ activated _blocks(5) 

fitness(5) = ^ 

tot _ blocks 

• Subsequently, GIP-PI is targeting a specific block T and the goal is to generate a 
sequence S able to cover it. In this phase sequences are targeted and the executed 
blocks are weighted by their correlation probability with respect to the target, 
measured as the weighted average of the execution counts of the basic blocks in 
the input cone (taking into account both control and data dependencies, thus 
potentially spanning several processes) of the target. The adopted weights take 
into account the probabilities of conditional execution that were statically 
computed in the database. 

fitness(5, T)= ^ correlation(h, T) 

be covered_ 66 ( S ) 

In both phases, a saturation mechanism prevents easy-to-execute but not-so- 
relevant statements from diverting the attention of the GA. 

Moreover, during each run of the GA, an heuristic mechanism detects individuals 
that may be useful in a subsequent run. When such an individual is found, it is saved 
and later it is inserted in the initial population of the correct run of the GA. These 
predefined individuals may never exceed 5% of the initial population. 

The GA in GIP-PI evolves a population of p individuals and in each generation X 
new sequences are first generated, then selection is performed on the whole set of 
p-i-X individuals. Individuals are selected for reproduction using a roulette wheel 
mechanism based on their linearized fitness. In p of the cases, the new individual is 
built mutating a single parent: the original sequence can be shortened, or enlarged, or 
some bits may be flipped. In \-p of the cases, the new individual is built mating two 
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different parents: the offspring sequence can inherit the beginning from one parent 
and the end from the other, or some entire bit column from each parent. 



3 Experimental results 

To test the effectiveness of the tool in generating test benches, we selected a set of 
VHDL benchmarks from the ITC 99 benchmark set [1TC99]. The first columns report 
some data about the RT-level descriptions, in terms of VHDL lines, VHDL processes 
(with hierarchy unflattened), and overall number of extracted basic blocks (BB). To 
have a better idea about circuit size, some characteristics of the synthesized netlists 
are reported in the last columns: number of Primary Inputs, Primary Outputs, Flip- 
Flops, and combinational gates. 

These benchmarks have been publicly released in September 1999 at the IEEE 
International Test Conference, and there are no published results, yet, to compare 
with. We compared our results against a pure random approach, to evaluate the 
effectiveness of the GA, but it was so easily overcome that results are not reported. 

The implementation consists of about 4,700 lines of C code for VHDL code 
analysis and instrumentation, linked to the LEDA LPI interface [LEDA95], and of 
2,700 lines of C code for the Genetic Algorithm and the interface to the simulator. All 
experiments were run on a Sun Ultra 5 running at 333 MHz with 256MB of memory. 

We adopt a population of p = 30 individuals, with X = 20 new individuals in each 
generation. The mutation probability was set top = 0.3. 
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Table 1: Benchmark characteristics 



In Table 2, we report the experiments we obtained with our prototypical tool in 
terms of percent number of covered branches, number of generated vectors, and 
required CPU time. These data demonstrate that for most descriptions, our method is 
able to reach a complete or very high branch coverage. There are a few circuits (e.g., 
b05 and bl2) where the obtained coverage is low: this is due to the specific 
characteristics of these circuits, which include highly nested conditional statements 
that the current version of our algorithm can hardly go through. 

It is worth noting that a manual analysis of the branches left uncovered proved that 
many of them were effectively unreachable, in most cases due to else or default 
statements that are required by the synthesis tool not to infer sequential logic, but are 
redundant since all cases have already been considered in previous tests. 
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b09 


100,00 


958 


511.5 


blO 


100,00 


364 


122.1 


bll 


94,59 


1,222 


1,410.2 


bl2 


36,44 


155 


1,022.3 


bl3 


100,00 


3,303 


4,203.9 


bl4 


93,03 


4,597 


7,875.5 


bl5 


91,81 


2,838 


9,369.2 


b20 


93,48 


7,784 


27,286.5 


b21 


93,69 


6,376 


28,878.9 



Table 2: Experimental results 



4 Conclusions 

This paper presented an automatic input pattern generation tool able to assist 
designers in the generation of a test bench for difficult parts of small- or medium- 
sized digital protocol interfaces. The approach resorts to a Genetic Algorithm that 
interacts with a simulator to generate new sequences able to increase the coverage of 
the test bench with respect to a predefined validation coverage metric. 

The methodology has been tested on the new ITC 99 benchmark set [ITC99]. 
Experimental results prove that the method is able to increase the quality of the 
validation process both over manual simulation and pseudo-random sequence 
generation. However, the proposed method could be easily extended to deal with any 
simulable descriptions, like the ESTELLE specification of a network protocol. 

The tool results have also been useful as a feedback for better understanding the 
most difficult parts of the design from the validation point of view. 
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c . h d t ction of intrusions o r comput r n t orks can b 
cast to th task of d t cting anomalous patt rns of n t ork traffic. In 
this cas , patt rns of normal traffic ha to b d t rmin d and corn- 
par d against th curr nt n t ork traffic, ata mining s st ms bas d 
on n tic Igorithms can contribut po rful s arch t chniqu s for th 
acquisition of patt rns of th n t ork traffic from th larg amount of 
data mad a ailabl b audit tools. 

In this pap r compar mod Is of data traffic acquir d b a s st m 
bas d on a distribut d g n tic algorithm ith th on s acquir d b a 
s st m bas d on gr d h uristics. Iso discuss r pr s ntation chang 
of th n t ork data and its impact o r th p rformanc s of th traffic 
mod Is. 



tr d cti 

h ris i th mb r of comp t r br a -i s, irt all occ rri g at a sit , 
d t rmi s a stro g r q st for ploiti g comp t r s c rit t ch iq s to pro- 
t ct th sit ass ts. ari t of approach s to i tr sio d t ctio do ist 
[ i g, 9 7]. om of th m ploit sig at r s of o attac s for d t cti g 
h a i tr sio occ rs. h s th ar bas do a mod 1 of possibl mis s s of 
a r so rc [K mar a d pafford, 994] . th r approach s to i tr sio d t ctio 
charact ri th ormal sag of th r so rc s dr mo itori g. i tr sio is 

s sp ct d h a sig i ca t shift from th r so rc ’s ormal sag is d t ct d. 

his approach s ms to b mor promisi g b ca s of its pot tial abilit to d - 
t ct o i tr sio s. Ho r, it also i ol s major chall g s b ca s of 
th d to acq ir a mod 1 of th ormal s g ral o gh to alio a thori d 
s rs to or itho t raisi g alarms, b t sp ci c o gh to r cog i d a tho- 
ri d sag s [ a ad rodl , 997, hosh t ah, 999, t ah, 999]. or 

th scop of this or , foe s to th tas of d t cti g i tr sio s b a al i g 

logs of t or s traffic. 

a asp cts of d plo i g i -practic approach s bas d o data mi i g, 
ho r, ar still op [a ad rodl , 99 ]. 

CO c trat h r o s 1 cti g i formati data r pr s tatio a d classi - 

catio p rforma c s. s 1 ar i g m thods, ploit d t o r 1 bas d s st ms: 

a h ristic o , I [ oh , 995], a d a stochastic o (bas dog tic 

S. Cagnoni et al. (Eds.): EvoWorkshops 2000, LNCS 1803, pp. 214-223, 2000. 
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algorithms), [ iorda aa d ri, 995, ri a d aitta, 996]. h rst 

s st m has b s 1 ct d b ca s of its pr io s s [ t ah, 999]; it ill 
th s act as b chmar . h s co d s st m has b s 1 ct d b ca s its i - 
tri sicall stochastic b ha ior a d sho Id alio th acq isitio of mor rob st 
mod Is [ ri a d aitta, 996]. 

I th folio i g, a d scriptio of th s st m ( ctio 2) a d of th 

p rim ts p rform d i th I ad co t ts ( ctio Sad ctio 

4) ar r port d. i all , th co cl sio s ar dra 

The Systems Regal and Ripper 

or spac r aso ill pro id h r a abstract d scriptio of both 1 ar - 

i g s st ms ad I as th ir f 11 d scriptio s ha air ad b 

p blish d. h s st m is a ailabl for fr from th a thor. 

[ iorda a a d ri, 995, ri a d aitta, 996] is a 1 ar i g s s- 
t m, bas do a distrib t d g tic algorithm ( ). It ta s as i p t a s t of 

data (trai i g i sta c s) a do tp ts a s t of s mbolic classi catio r 1 s char- 

act ri i g th i p t data, s s al, 1 ar i g is achi d b s archi g a spac of 

ca didat classi catio r 1 s. 

h la g ag s d to r pr s t classi catio r 1 s is a Hor cla s la g ag 

i hich t rms ca b ariabl s or disj ctio s of co sta ts, a d gatio oc- 

c rs i a r strict d form [ ichals i, 9 3]. ampl of a atomic pr ssio 

CO tai i g a disj cti t rm is r( r ), hich is s ma ticall 

q i al t to r( ) r r( r ). ch form las ar r pr s t d 

as bitstri gs that ar act all th pop latio i di id als proc ss d b th 
lassical g tic op rators, op rati go bi ar stri gs, ith th additio of 
tas ori t d s a ad ra crosso rs ar ploit d, i a adap- 
ti a , i sid th s st m (for d tails s [ iorda a a d ri, 995]. 

is a distrib t d g tic algorithm that ff cti 1 combi s th h - 

or of ich s a d p ci s of iological ol tio tog th r ith parall 1 proc ss- 
i g. h s st m archit ct r is mad bast of t dd impl tic Igo- 

rithms ( ) [ oldb rg, 9 9], hich coop rat s to si ad scriptio spac , 

a d b a p r isor proc ss that coordi at s th s fforts b assig i g to 
ach of th m a diff r t r gio of th ca didat r 1 spac to b s arch d. I 
practic this is achi d b d amicall d isi g s bs ts of th datas t to b 

char act ri d b ach 

h s st m I [ oh , 995] is bas do th it rat d applicatio of 

a gr d h ristic, similar to th I formatio ai m as r [ i la , 993], to 

b ild CO j cti classi catio r 1 s. t ach it ratio , thos trai i g i sta c s 

corr cti classi d b th fo d r 1 s ar r mo d a d th algorithm co c trat 

o 1 ar i g a classi catio r 1 for th r mai i g o . h s st m o tp ts a 

ord r d list of classi catio r 1 s (possibl associat d to ma class s) to b 
appli d i that sam ord r to classif a i sta c . i t r sti g f at r s of 
th m thod is that it ploits o -li r 1 pr i g hil i cr m tall b ildi g 

a classi catio r 1 to a oid o r tti g. 




216 F. Neri 



3 tr si d t cti it r ti p r ti 

t t c t st 

al atio of o r a i tr sio d t ctio tas b ploiti g data 

from th I formatio ploratio hooto t roj ct (I ) is r port d i this 

s ctio . h I mad a ailabl t or logs prod c d b ’tcpd mp’ for al- 

ati g data mi i g tool o r larg s t of data, h s logs r coll ct d at th 

gat a b t a t rpris a d th o tsid - t or (I t r t). I th 

I CO t t, d t cti g i tr sio s m a s to r cog i th possibl occ rr c of 

a thori d (’bad’) data pac ts i t rl a d ith th a thori d (’good’) o s 

0 r th t or dr mo itori g. h I ’s proj ct ma s a ailabl fo r t- 

or logs: o is g ara t ot to co tai a i tr sio att mpts, h r as th 

oth r o s do i cl d both ormal traffic a d i tr sio s att mpts. I th I 

CO t t, o classi catio for ach data pac t is r q st d, i st ad a o rail 

class! catio of a b ch of th t or traffic, as co tai i g attac s or ot, is 
d sir d. approach to i tr sio d t ctio , bas do a omal d t ctio , has 
b s 1 ct d. 

proc d as folio s. I data ca b partitio d, o th basis of th ir I 

addr ss s, i to pac ts iti g th r f r c i stallatio ( tgoi g), t ri g th 

1 stallatio (I corn! g) a d broadcast d from host to host i sid th i stallatio 

(I t rla ). hr mod Is of th pac t traffic, o for ach dir ctio , ha b 

b ilt from th i tr sio -fr datas t. h , th s mod Is ha b appli d to 
th thr datas ts co tai i g i tr sio s. p ct to obs r a sig i ca t ari- 
atio i th classi catio rat b t i tr sio -fr logs a d logs co tai i g 

i tr sio s b ca s of th a r a charact ristics of th traffic prod c d b th 

i tr si b ha ior. If this o Id act all occ r, co Id ass rt that th 1 ar d 

traffic mod Is corr cti capt r th ss tial charact ristics of th i tr sio -fr 

traffic, ot that h th r this approach sho Id or , co Id co cl d that 

a i tr si att mpt is happ i g b ca s of a diff r t a tra att r . 

p rim ts ha b p rform d both ith I ad 

hi is appli d to th I data, th classi catio rat app ari g 

i abl b com s id t [ t ah, 999]. ach tabl tr r pr s ts a clas- 
si catio rror as m as r d o r th fo r t or logs, o for ach li , a d 

ith r sp ct to th thr class lab Is: tgoi g, I t rla , a d I comi g. h 

corr ct classi catio rat s ca b obtai d b s btracti g a tr ’s al from 

h s th rst ro sho th misclassi d ormal traffic pac ts, h r as th 
oth r o s sho s th misclassi d pac ts d ri g a i tr sio att mpt. 

h s r s Its ha b obtai d b appl i g I to th data as a ail- 

abl fromth tcpd mp d 1 s (s pp di ). opr proc ssi go rth data, 
s ch as f at r co str ctio , has b appli d. h p rim tai di gs sho s 
that th acq ir d mod Is do ot hibit r diff r t classi catio rat h 

appli d to logs CO tai i g i tr sio s ith r sp ct to i tr sio -fr logs, h s 

di gs ma s gg st that th ploit d data r pr s tatio is too d tail d ith 

r sp ct to th capabilit of th 1 ar i g s st m. I t r , this ca s s th 1 ar d 

mod Is to miss th i formatio charact ri i g i tr sio -fr traffic. 
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. p rim ntal r suits of appl ing I to I datas ts using th ra data 

r pr s ntation. ach tabl ntr stat s a classification rror. 



at as t 


int rlan 


incoming 


outgoing 


normal traffic 


. 4 


. 4 


. 4 


intrusion 


.23 


. 7 


. 4 


intrusion2 


. 9 


. 7 


. 5 


intrusions 




. 4 


. 4 



. p rim ntal r suits of appl ing I to I datas ts using a compr ss d 

data r pr s ntation. 



at as t 


int rlan 


incoming 


outgoing 


normal traffic 


. 2 


. 5 


. 4 


intrusion 






.2 


intrusion2 


. 3 


. 3 


. 2 


intrusions 




.2 


. 2 



olio i g this obs r atio , d lop a mor compact r pr s tatio for 
th pac ts that co sists i mappi g a s bs t of f at r ’s al s i to a si gl 
al , th s r d ci g th card! alit of possibl f at r s al s (s pp di 
). ploiti g this r pr s tatio , I ’s p rforma c s b com th o s 
r port d i abl 2 a d ’s p rforma c s ploiti g th sam compact 

data r pr s tatio app ar i abl 3. h obs r d g r s sho a mor stabl 

class! catio b ha ior of th mod Is across diff r t traffic co ditio s. Iso a 
mor disti ct class! catio p rforma c b t th i tr sio -fr log a d th 
logsi cl di g i tr sio sis id t. compr ssio -bas dr pr s tatio is th a 
al abl a of i cr asi g class! catio p rforma c s itho t i trod ci g com- 
pl f at r that ma i ol s additio al proc ssi go rh ad. al atio of 

th ff ct ca s d b th additio of compl f at r s to th ra t or data 

r pr s tatio has b p rform d i [ t al., 999]. 

or th sa of clarit , a ampl of r 1 charact ri i g i tr sio -fr I - 
corn! g pac ts, 1 ar d b , app ars i ig r .hi corn! g pac ts 

ar charact ri d i t rm of th al s of th f at r s from th ir /I h ad r. 

his r 1 s cc ssf 11 co rs 7349 I comi g pac ts itho t b i g fool d b a 



. p rim ntal r suits of appl ing L to I datas ts using a compr ss d 

data r pr s ntation. 



at as t 


int rlan 


incoming 


outgoing 


normal traffic 


. 2 


. 4 


. 4 


intrusion 


. 2 


. 5 




intrusion2 


. 6 




. 2 


intrusions 


. 2 


. 5 
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I srcprt( ,[[ ,2 ],[4 , ],[ 5 ,2 ],[ 5 ]]) and 

dstprt( ,[ 24]) and flag( ,[ ,ptj) and 

s q ( ,[[ ,5 ],[2 ,3 ],[5 ,5 ],[ ]]) and 

s q2( ,[[5 , ],[2 ,3 ],[5 ,2 ]]) and 

ack( ,[[ ,3 ],[5 , ]]) and 

in( .[[ .2 ],[ 3 ]]) and 

bnf( ,[ =5 2]) 

H Incoming ack t( ) 

o rag : (Int rlan, Incoming, utgoing) = ( , 7349, ) 



i . . ampl of a rul charact ri ing part of th incoming traffic, h rul d scrib s 
7349 incoming pack ts ithout confusing th m ith an outgoing or int rlan pack t. 



I t rla or tgoi go s. d scriptio of th pr dicat s app ari g i th r 1 

is pro id d i pp di 

tr si d t cti it tr si 

t cti ti r r 

also p rform d a additio al al atio of o r approach or t- 
or logs from 99 I tr sio t ctio al atio rogramm 

[ ippma t al., 999] hos obj cti as to s r ad al at r s arch i 
i tr sio d t ctio . sta dard s t of data to b a dit d, hich i cl d s a id 

ari t of i tr sio s sim lat d i a militar t or iro m t, as pro id d. 

ploit d data a ailabl from th K ’99 I tr sio t ctio o t st . 
h ra trai i g data as abo t fo r gigab t s of compr ss d bi ar 

d mp data from s s of t or traffic, his as proc ss d i to abo t 

millio CO ctio r cords, imilarl , th t o s of t st data i Id d 
aro d t o millio co ctio r cords. co ctio is a s q c of pac - 

ts starti gad di g at som 11 d d tim s, b t hich data flo s 

from a so rc I addr ss to a targ 1 1 addr ss dr som 11 d d protocol. 

ach CO ctio is lab 1 d as ith r ormal, or as a attac , ith actl o 

sp ci c attac t p . ach co ctio r cord co sists of abo t b t s. ttac s 

fall i to fo r mai cat gori s: 

: d ial-of-s r ic , .g. s flood; 

2 : a thori d acc ss from a r mot machi , .g. g ssi g pass ord; 

2 : a thori d acc ss to local s p r s r (root) pri il g s, .g., ario s 

“b ff r o rflo ” attac s; 

rob : s r ilia cad oth r probi g, .g., port sea i g. 

I practic t o data 1 s co tai i g class! d co ctio s ar a ailabl : o has 

to b s d for acq iri g a mod 1 of th traffic a d th oth r o for t sti g its 



Information about K ’99 Intrusion t ction ont st is a ailabl on-Iin at 
http:// . psilon.com/kdd9 /task.html. 
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p rforma c s. h disti ctio is importa t b ca s th t st 1 co tai s attac 
t p s ot occ rri g i th 1 ar i g 1 . his is i t d d to ma th tas mor 
r alistic. I gr2ad gr3, p rforma c s of I pi s ta- ar i g 



0.4 



0.2 



0 



Probe 

Dos — 

U2r 

R2I 



0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 



i . . t ction p rformanc s hibit d b I plus ta-L arning on th 

t st data, n t nd d r pr s ntation of th data and a compl 1 arning 
approach (m ta-1 1 1 arning) ha b n ploit d. 



(as s d i [ t ah, 999]) ad or ’s data ar r sp cti 1 

sho . I this cas p rforma c s ar sho b ploiti g c i r p rati g 
r s as do h al ati g data mi i g tools. I th g r s, th a is r p- 

r s ts th fals alarm rat , i. . th p rc tag of ’ ormal’ co ctio s lab 1 d 

as i tr sio s, h r as th a is r pr s ts th d t ctio rat , i. . th p rc tag 

of i tr sio s that ha b corr ctl r cog i d. ach li is associat d to a dif- 

f r t attac t p . his i d of graph is s d to sho ho a classi rs b ha ior 

d grad s h ”r la i g” its matchi g co ditio s: g rail mor i tr sio ar 

d t ct d hil also co ri g mor fals alarms. Ho r i th cas of a s t of 

s mbolic classi catio r 1 s (it is act all b tt r to sa i th cas of I 

ad ) th r ar o co ditio s to b r la d that is th r aso of th 

flat li at th top. 

h r port d p rforma c s ha b obtai do th co ctio s occ r- 
ri g i th t st 1 . hr port d graphs sho similar d t ctio p rforma c s, 
b t th mod Is acq ir d b th s st ms, for rob a d mot - o- ocal 

( 21) attac s t p s. I st ad, ’s mod 1 p rforms slightl b tt r o 

t p attac s b t orst o s r- o- oot ( 2r) attac s. 

t CO sid r, o , th mod li g approach s ploit d b th t o s st ms. 
a d tolfo [ t ah, 999] r I ora tdd data r pr s tatio 
of th tcp CO ctio i cl di g, i additio to th basic tcp f at r s, d ri d 

i formatio s ch as: th mb r of co ctio s to th sam host i th past 
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0.4 



0.2 



0 



Probe 

Dos — 

U2r 

R2I 



0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 



i . . t ction p rformanc s hibit b L on t st data (no additional 

ta-L arning has b n us d). compr ss d data r pr s ntation has b n ploit d. 



t o s CO ds (’co t’), th mb r of co ctio s to th sam s r ic , as th 
c rr t CO ctio , i th past t o s co ds (’sr -co t’). h s f at r s ha 
b chos o th basis of th a thors p rtis . pr proc ssi g of th ra 
t or logs is r q ir d i ord r to ploits this f at r s. ral classi rs 
(r 1 s ts) for ach attac t p ha b obtai d. t all m ta-1 ar i g, 
i. . 1 ar i g at th classi r 1 1, has b appli d to prod c th r port d 

p rforma c s. 

,0 th CO trar , has b r aft r appl i g a compr ssio mappi g 
to th f at r al s, as d scrib d i pp di . 1 th basic f at r s of a 

CO ctio ha b co sid r d s ch as: ’d ratio stati g th 1 gth 

( mb r of s CO ds) of th co ctio , ’protocol-t p stati g th t p of th 

protocol ( .g. tcp, dp, tc.), or ’src-b t s’, stati g th mb r of data b t s 
from so rc to d sti atio . o additio al m ta-1 ar i g phas is c ssar . 

c si s 

r port p rim ts that sho th pot tialit of a distrib t d g tic 1 ar r 
to th mod li g of t or data. o diff r t s t- ps to d al ith d t cti g 
i tr sio s ha b plor d. 

a al da data pac t r pr s tatio ploiti g compr ssio of th f a- 
t r ’s al si th ffort to r d c th compl it of acq iri g mod 1 of th 
traffic. b li this b i g a importa t r q isit for th a tomatic mod li g 

a d th o -li d plo m t of i tr sio d t ctio s st m. 

h p rim tal r s Its s pport s of th compr ssio of th f at r al s 

as a al abl m thod to i cr as d t ctio p rforma c s hil a oidi g th s 

of d ri dad compl f at r s that i ol additio al comp tatio al o rh ad. 
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c d ts 

a tha s to th a o mo s r i rs hos s f 1 comm ts h Ip d m i 
impro i g th pap r. 

pp di . r ti p r ti t t r 

d t r pr s t ti 

h I data (a ailabl o li at http://iris.cs. ml. d ) ha b coll ct d 
b m a s of th tilit . a i g i to acco t pri ac co c r s, th 

data portio of ach pac t has b dropp d. or ach pac t i th datas ts 

th folio i g attrib t s ar a ailabl : 

tim - CO rt d to float! g pt s co ds .. hr*36 +mi *6 +s cs. 
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o this CO ctio . 

i - th mb rofbtsofrci bffr spac a ailabl from th oth r dir c- 
tio o this CO ctio . 

1 - if a dp pac t , th 1 gth. 

op - optio al i fo s ch as (df) ... do ot fragm t. 

artic lar att tio has to b ta h d ali g ith Ids li ’op’ that co - 

tai s a larg amo t of al s. 

pp di . c pr ss d t r r pr s t ti 
d t 

om f at r s of th I data ma ass m a larg s t of al s ith r co ti - 
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th 1 ar d mod Is b ca s of th i tri sic difflc It of acq iri g r 1 ha i g 

a g ral scop . h , a r d ctio of th ra g of pot tial al s is d sirabl 
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comp tatio al compl it . 

alt r ati approach to this probl m co sists i addi g/b ildi g mor 

compl f at r s, combi i g th basic o s, to th origi al data r pr s tatio . 

do ot folio this approach i this or , b ca s b li that th pr io s 
approach is simpl rad sho Id b th rst to b a al d. 
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Abstract. Previous publications hy the authors have demonstrated a bimodal 
performance profile for simple evolutionary search on variants of the Adaptive 
Distributed Database Management Problem (ADDMP) and other problems over 
a range of evaluation limits. This paper examines an anomaly seen in one of 
these profiles and together with results from a range of other problems, shows 
that with sufficiently high evaluation limits, a multimodal performance profile is 
apparent in search spaces with significant numbers of deceptive local optima. 
This is particularly apparent in the performance profile of the Hierarchial If and 
only If problem (H-IEE) where the regular structure of the search space 
produces several distinct peaks and troughs in the performance profile, possibly 
indicative of a range of specific ‘fitness barriers’ which are surmountable by 
specific rates of mutation. This observation could prove important in general 
EA parameter tuning over a range of problems with similar characteristics. 
Further, the existence of optimal mutation rates inducing a minimum in standard 
deviation of run-time, is of critical importance in the application of EAs to real- 
time, real-world problems. 



1 Introduction 

Many real world, real time applications of Evolutionary Algorithms (EAs) [1,3,4, 6] 
require the search process to reliably produce quality solutions in a fixed number of 
evaluations. To increase the likelihood of this, it is often essential to tune the 
algorithm by selection of suitable parameter values such as population size and 
mutation rate. One such application which has been studied extensively by the authors 
[11,12,13,14,15,16,17] is the Adaptive Distributed Database Management Problem 
(ADDMP) which attempts to balance a number of user loads onto a range of available 
servers over a communications network to maximise a given quality of service metric. 
The problem is reduced to one of combinatorial optimisation, where a ‘solution 
vector’ generated by the EA defines for each client node (determined by locus), which 
server to use (determined by allele value) using a natural k-ary representation (where 
alleles are integers in the range 1 through k). This effectively determines a route 
through the communications network, and the combined delay imposed by the server 
processing time and the communications latency is calculated by a performance model 
utilising the principles of MMl queuing and Little’s Law. The problem has been 
likened to a form of complex ‘bin-packing’ and further details are available in [12,13] 
with example code and datasets available at http://www.dcs.napier/ac/uk/evonet part 

S. Cagnoni et al. (Eds.): EvoWorkshops 2000, LNCS 1803, pp. 224-234, 2000. 
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of the ECTELNET website (ECTELNET is the Telecommunications subgroup of the 
European Network of Excellence in Evolutionary Computation) 



Client : 
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10 


Which Server to use : 
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3 


2 


2 


3 


4 


1 



Fig. 1. Example solution vector. 

Many instances of the ADDMP exist, with different user load patterns, different server 
profiles and different communications network topologies, and some of these are 
discussed in [13]. Some of these instances have been shown to present considerable 
difficulty to a range of algorithms [11,14] and in general, an Evolutionary algorithm, 
correctly tuned, has been shown to give the most consistent performance. Previously 
published studies of one of these instances [16,17] show a three dimensional 
performance profile for a simple generational Breeder EA [7] using 50% elitism, 
uniform crossover [19], and uniformly distributed allele replacement mutation at a 
fixed rate per gene. Experiments were run over a wide range of mutation rates, from 
lE-7 to 0.83, the latter effectively degenerating the EA into random search, and a 
wide range of population sizes typically 10 to 500 members in steps of 10. Each run 
was allowed 20,000 evaluations and each experiment was repeated 50 times. During 
each run, a note was made of the fitness of the best solution found, and the evaluation 
number at which it was first found. For the 50 runs all with the same experimental 
parameters, the mean of the ‘time to best solution’ was plotted against population size 
and mutation rate, and the resulting graph is shown in Figure 2. As was commented on 
in [16], this produces, for a given, low population size, a bimodal performance profile, 
where the number of evaluations taken to first find the best solution to be found in the 
run first rises, then falls, then rises again before falling into random search. These 
features are interpreted as follows : the low values of evaluations used at low mutation 
rates represent premature convergence on poor solutions, rapidly exhausting the 
limited diversity available in the initial population which cannot be supplemented by 
mutation due to its low rate; as mutation rates increase, the EA is able to progress 
further with its search, as depleted allele values are re-injected into the population 
with increasing frequency by the increasing mutation rate; a point is reached where 
there is a sufficiently high rate of mutation to allow the search to typically utilise in 
excess of 80% of the available evaluations in terms of fitness improvement - this is 
the left hand peak on Figure 2; further increases in mutation rate allow good solutions 
to the problem to be found in fewer evaluations, leading to a point where the mutation 
rate allows good solutions to be found in a minimum of evaluations - the trough in 
Figure 2; as mutation rate is increased further, performance deteriorates as the 
mutation rate begins to destroy progress almost as fast as it is made - the right hand 
peak on Figure 2; finally at very high rates of mutation, in excess of 40%, 
performance deteriorates to random search with the mean number of evaluations used 
tending to 50% of those allowed. This interpretation is supported by examination of 
the mean fitnesses of solutions found as mutation rates are increased which show : 
poor fitness at low mutation rates; a steady improvement in fitness for mutation rates 
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above the first peak in Figure 2; a plateau of good fitness coinciding with the trough of 
minimal evaluations in Figure 2; and a deterioration in fitness as mutation rates 
approach the right hand side of Figure 2. This co-incidence of trough in evaluations 
and peak in fitness occurs at mutation rates around 1 / L (where L = binary 
chromosome length, here equivalent to around 2.5%), which is in support of 
theoretical studies by, amongst others, Baeck [1], Muhlenbein [8] and many empirical 
studies including [2,8,9,18] by Deb et al, van Nimwegen and Crutchfield, Oates et al, 
and this and other features are discussed in more detail in [16,17]. 

Of particular interest however in this paper is the anomaly which can be seen on the 
left hand edge of the right hand peak in Figure 2 at low population sizes (towards the 
back of the figure). Here, the rise in mean evaluation number with increasing mutation 
can be seen to be not smooth, with a ridge feature apparent over a range of lower 
population sizes. Such an anomaly was not seen at lower evaluation limits, nor on 
extensive investigations of simple ‘unimodal’ search spaces such as presented by the 
‘Max-Ones’ problem. The remainder of this paper explores this phenomenon in more 
detail, with section 2 describing three other test problems and their experimental set 
up; section 3 describing the results at both 20,000 and 1 million evaluations; section 4 
drawing some preliminary conclusions from these results and sections 5 and 6 
acknowledging support and referenced material. 



2 Method 

To examine this anomaly in more detail, a series of experiments was conducted with a 
range of ‘standard’ test problems over similar ranges of experimental parameter 
values, and a second series of experiments conducted with a much higher evaluation 
limit and more refined scale of mutation rates. 

The first problem to be looked at was the ‘Max-Ones’ problem where in a binary 
string, fitness is calculated to be the number of ‘I’s present in the chromosome. With 
standard representation, steady state, single 3 way tournament selection [3], uniform 
crossover and simple per gene mutation, this presents a simple unimodal search 
landscape which can be ascended by even the simplest ‘hillclimbing’ algorithm. 50 
trials were run for each parameter setting, which ranged from population sizes of 2 to 
100 and per gene mutation rates from lE-7 to .83. Mean evaluations to first find the 
best solution found in 20,000 evaluations were noted. Chromosome length for results 
shown here was 50 bits, however similar experiments have also been done at 33, 300 
and 1630 bits, giving corresponding results. 

The second problem to be investigated was a 64 bit implementation of Watson’s H- 
IFF (Hierarchical If and only If) problem [20,21], which increasingly rewards ever 
larger aligned blocks of contiguous ‘I’s or ‘O’s and can be represented as : 
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1, iflBI=l 

/(B) = IBI +/(Bl) +/(Br), if(IBI>l)and(Vi{b, = 0}orVi{bi=l}), 

/(Bl) +/(Br), otherwise 

where B is a block of bits, {bi, b2, ... bn}, IBI is the size of the block=n, bi is the ith element of B, and Bl 
and Br are the left and right halves of B (i.e. Bl = {bi, ... bn/2}, Br = {bn/2+1, •• .bn}. N must be an integer 

power of 2. 

H-IFF therefore has two global optima, one at all ‘I’s and one at all ‘O’s. There are 
secondary optima at strings of 32 ‘I’s followed by 32 ‘O’s and vice versa, and a range 
of sub-optima at combinations of aligned blocks of ‘I’s and ‘O’s each of length 16, 8, 
4 and 2. The ‘search landscape’ can be considered to be rugged but in a highly 
structured fashion. Simple hillclimbers perform inadequately on this surface as the 
basin of attraction of the global optima is very small, and there are many local optima 
when seen by single point mutation alone. Even standard one-point and two-point 
crossover operators have been seen to require very high population sizes to achieve 
reasonable and consistent performance on this problem unless an appropriate diversity 
maintenance technique is employed. 

The third problem reported on is this paper is a 50-8 Kaufman NK landscape [5] 
generated by defining a table of random numbers of dimension 50 by 512. For a 
binary chromosome of length N=50, starting at each of the 50 gene positions, K+1 
(here 9) weighted consecutive genes are used to generate an index (in the range 0- 
511), and the 50 values so indexed from the table are summed to give a fitness value. 
For NK landscapes with a K value of 0, each gene position contributes individually to 
fitness, hence producing a unimodal landscape to single point mutation hillclimbers. 
However as K is increased, any single point mutation will affect K+1 indexes, rapidly 
introducing positional linkage into the problem and producing a rugged and 
increasingly unstructured landscape. 

The ADDMP, Max-Ones, H-IFF and NK50-8 problems were all trialled at 20,000 
evaluations in the first series of experiments, with results shown in Figures 2,3,4 and 5 
respectively. A second series of experiments was then performed with a limit of 
1,000,000 evaluations, for ADDMP (Figure 6) over a range of population sizes (10- 
300 in steps of 10) and for Max-Ones and H-IFF at a fixed population size of 20 
(Figures 7 and 8). For easier comparison of results, the ADDMP 1,000,000 evaluation 
runs at a population size of 20 are shown more clearly in Figure 9. Plots showing plus 
and minus one standard deviation seen over the 50 trials and mean fitness are also 
shown in each of the fixed population size graphs. 

Unless otherwise stated, all experiments (with the exception of ADDMP at 20K evals) 
were conducted with a steady state, 3 way single tournament GA using uniform 
crossover. The H-IFF experiments were conducted using one-point crossover due to 
the obvious inappropriateness of the uniform operator on this problem. 





Fig. 3. One Max at 20K evaluations 



Fig. 5. NK 50-8 at 20K evaluation 









Fig. 7. One Max at IM evals, pop size = 20 Fig. 9. ADDMP at IM evals, pop size = 20 
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3 Results 

An interpretation of Figure 2 has already been given in Section 1 of this paper, and a 
similar performance profile at 20,000 evaluations can also be seen for the Max-Ones 
problem in Figure 3. Again, at low population sizes and low mutation rates, the search 
is seen to stall after very few evaluations, as diversity in the initial population is 
rapidly depleted. Very low rates of mutation prevent re-occurrence of potentially 
useful alleles and thus the mean number of evaluations used is very low. As mutation 
rates increase, the search is able to progress to higher mean numbers of evaluations, 
until a point is reached where mutation rates allow, on average, most of the 
evaluations to be utilised in the search for good solutions. As mutation rates increase 
further, these good solutions are found in less and less evaluations, until the trough 
point in Figure 3 is reached where global optimum solutions are found in the 
minimum number of evaluations. Beyond this rate of mutation, performance 
deteriorates eventually degenerating into random search. However in contrast to 
ADDMP (Fig 2), this can be seen to deteriorate smoothly, with no anomaly on the left 
hand edge of the right hand peak. The mutation rate inducing the first peak can be 
seen to be effectively independent of population size (over its visible range) and 
occurs at a rate around .0001. The trough rate is centred around a mutation rate of .05 
(= 5% chance of applying a New Random Allele (later referred to as NRA), this could 
be written as 2.5% chance of applying ‘guaranteed flip’ mutation ~ \ ! 50). 

Figure 4 shows the performance profile for the 64 bit H-IFF problem again at 20,000 
evaluations. Here, in stark contrast to Figure 3, a third peak can be seen in the 
performance profile, particularly at low population sizes. It is believed that the left 
hand trough corresponds to a low mutation rate which represents a lower bound that is 
useful for finding a certain set of sub-optima in the problem space - for example 
blocks of size 4. However, as the mutation rate is increased, this rate becomes sub- 
optimal for this block size, and another mutation rate is found which proves useful in 
finding larger block sizes - possibly of size 8 or 16. This supposition is supported by 
the fact that a plot of mean fitnesses found against mutation rate shows marked step 
improvements between each of these troughs in evaluations used, implying that the 
EA was suddenly able to find better optima at each of these specific mutation rate 
boundaries. This observation can be seen in Figure 8 and is now the subject of further 
investigation which will be reported on in due course. It is likely that there exist other 
‘optimal’ mutation rates for other block sizes in this problem, but these are not 
distinguishable at this scale of resolution on the mutation rate axis and this is again 
further explored in the discussion of Figure 8. As mutation is increased beyond an 
upper threshold, performance is seen to deteriorate, as in the previous two problem 
cases, into random search. The mutation rate inducing the first and second peaks are 
seen to be .0001 and .013 respectively. 

Figure 5 shows the 20,000 evaluation performance profile for the NK50-8 problem. 
Here it can be seen, as was also visible in the ADDMP profile (Figure 2), that the 
deterioration of performance from the optimum trough is not smooth, with a ‘ridge- 
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like’ anomaly again present on the left hand edge of the rightmost peak over a range of 
lower population sizes. 

Figure 6 shows a performance profile for the same ADDMP instance as Figure 2, 
however this time the EA is allowed 1,000,000 evaluations over a population range of 
10 through 300 in steps of 10. In fact results were obtained at each of fifty thousand, 
one hundred thousand, two hundred thousand and five hundred thousand evaluations, 
with the observed anomaly simply becoming more distinct as evaluation limit 
increased. Indeed, the mutation rate inducing the first peak, first trough and anomaly 
features were seen to decrease with increased evaluation limit, implying a relationship 
to total number of mutations utilised as well as actual mutation rate. This observation 
is also currently the subject of further investigation and will be reported on in due 
course. 

As can clearly be seen in Figure 6, the mutation rate inducing the first peak has 
dropped from 4E-4 at 20,000 evaluations (Eigure 2) to 1.3E-5 at 1 million evaluations. 
The secondary and tertiary peaks ‘emerged’ from the left hand edge of the right hand 
peak with higher evaluations limits until here at 1 million evaluations they can clearly 
be seen to be performance landscape features in their own right, persistent over a wide 
range of population sizes. Indeed, this plot shows similarities with results from the H- 
lEE problem (see Eigures 4 and 8). 

Eigure 7 shows the 2 dimensional plot at a fixed population size of 20 on the ‘Max- 
Ones’ problem at 1,000,000 evaluations. Here, the position of the first peak can be 
seen to be at a considerably lower mutation rate than in Eigure 3. The trough of low 
evaluations is seen to be much wider, indicating a wider range of mutation rates 
capable of finding the global optimum in a low number of evaluations. Eigure 7 also 
shows plots of the mean number of evaluations plus and minus 1 standard deviation of 
results over the 50 runs. As can be seen, where mutation rates are too low, process 
variation is high, but as mutation rates approach optimum (the right hand edge of the 
left hand peak), this variation is reduced. Once mutation rates become counter 
productive (above 20% NRA), diversity is seen to increase again as the EA 
degenerates into random search. Also shown is the mean fitness of the best solution of 
each of the 50 runs at each mutation rate. The plateau representing runs finding the 
global optimum 50 times out of 50 is seen to coincide with the trough in evaluations 
indicating an ideally tuned EA. 

Eigure 8 shows a 2 dimensional performance profile at 1 million evaluations on the H- 
lEE 64 problem with a population size of 20. Here 3 distinct peaks and troughs can be 
seen before deterioration into random search, with specific peaks at a mutation rate of 
lE-6, another at 1.6E-3 and a third at 4.4E-2. This is an increase in the number of 
features seen at 20,000 evaluations (Eigure 4), and again it is noted that the features 
occur at lower mutation rates given the higher evaluation limit. It can also be clearly 
seen on the left most two troughs that the standard deviation in the troughs is at a 
distinct minimum on the left hand side of each trough in evaluations. Also, the average 
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fitness plot clearly shows marked increases between the troughs showing that the EA 
is able to find step-like improvement as mutation rates are increased. 

Finally, Figure 9 shows the 2 dimensional performance profile for our ADDMP 
scenario at 1 million evaluations with a population size of 20. Whilst the first peak and 
trough are clear (at mutation rates of 1.3E-5 and 8.2E-4 respectively, the second and 
third peak are less distinct, although clearly exist as features in Figure 6. There are 
clearly regions of low standard deviation corresponding to troughs at 8.2E-4, 2.5E-3 
and 0.2, and a region of higher mean and much higher deviation between mutation 
rates of 2.5E-3 and 0.16. This relative disorder is to be expected as any structure in 
the search space is likely to be far less regular in the ADDMP scenario than in the 
case of H-IFF. Nonetheless, Figures 6 and 9 clearly show there to be some features for 
which key rates of mutation are either highly optimal or highly sub-optimal. The plot 
of mean fitness does not show such clearly defined ‘step-like’ improvement as in 
Figure 8, however once again it can be seen that there is general improvement towards 
higher mutation rates. Interestingly, and in contrast to other results shown here, fitness 
deterioration at exceedingly high rates of mutation is not as marked, again suggesting 
a lack of structure in the search space, favouring a ‘random search’ like process. Of 
course the marked increase in both mean and standard deviation of the number of 
evaluations taken to find these good solutions shows the relative inefficiency of 
random search with respect to a well tuned EA on this problem. The ideal mutation 
rate here being 0.31 (NRA) inducing high fitness solutions, in a minimum of 
evaluations with a minimum of process variation. 



4 Conclusions 

For the simple unimodal search space explored here, it can clearly be seen that a 
moderate rate of mutation (usually close to 1 / L) can produce optimal performance 
both in terms of consistent quality of solution found and minimum number of 
evaluations taken to find them. Where a higher number of evaluations is allowed, this 
range of suitable mutation rates extends towards ever lower values. 

For the structured multi-modal search space, there appear to be a series of ‘optimal’ 
mutation rates, capable of exploiting certain features of the search space. In the case 
of H-IFF this is likely to be aligned blocks of constant length each separated by equal 
Hamming distance. Values around these ‘optimal’ rates are seen to be clearly sub- 
optimal in terms of process repeatability (increased mean and standard deviation). A 
‘globally optimum’ mutation rate again seems to exist close to 1 / L which gives 
optimum fitness solutions, but in this case, not in an optimally minimum number of 
evaluations (however it is possible that further experimentation with a finer grained 
mutation rate range may show this not to be the case). 

The performance profile of the ADDMP has been shown to exhibit some features in 
common with both structured and unstructured multi-modal search spaces. It is clearly 
multimodal over a range of population sizes where sufficient evaluations are 
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permitted. The existence of key mutation rates inducing a low standard deviation in 
number of evaluations is highly significant when considering the application of EAs to 
this problem domain allowing a high degree of confidence in process repeatability. 
The fact that once again a mutation rate close to 1 / L is seen to give high fitness 
solutions in a low number of evaluations with low standard deviation, even when the 
allowed range of evaluations was considerably higher, is of critical importance to the 
real time application of these techniques to a real world problem domain. 
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c he nstructi n f transp rt pr t c Is hich ff r r liabl 
c unicati n is a c plicat d task, h c unicating ag nt ust 
quickl adapt t chang s in th n t rk in rd r t aintain pti al 

p rf r anc . his adapti 1 nt is an tr 1 difficult c p n nt 

t c nstruct as th highl d na ic n ir n nt in hich th pr t c 1 
p rat s is difficult t pr diet, ur rk att pts t aut at th d sign 
pr c ss b c n rting it fr ad sign pr bl t n f pti isati n, in 

hich g n tic alg rith s ar us d t s arch th spac f p ssibl pr t c 1 
d signs in an att pt t find th pti al s luti n. pr s nt r suits 
fr p ri nts in hich ha 1 d alt mating bit pr t c Is and 

als ind d fl c ntr 1 pr t c Is, hich ha high chann 1 utilisati n. 

r c i 

is ork plor s t possibilit of appl i g ol tio ar s arc strat gi s 
to t s t sis of t ork co icatio s protocols. pri cipl s a 

a opt r origi all pio r i t 1 of ol tio ar obotics [3] a [2] 
ic s s g tic algorit s to s arc t sig spac of robot co trol arc i- 

t ct r s for s st s ic ibit a sir b a io r. Iti at goal of o r 

r s arc is to lop a r sabl t o olog or co c pt al fra ork for t 
CO str ctio of t a apti algorit s a protocols plo b tra sport 

la r protocols for pack t s itc t orks. p ci call ar foe si g o t 

lop t of CO g stio CO trol a a oi a c algorit s s c as t os 
plo b r liabl protocols s c as t ra sport o trol rotocol( ). i c 
is so t si 1 s it is cl ar t at a s all i pro t i its abil- 

it to tilis r so rc s a a apt to c a g ill gr atl i pro global t ork 

p rfor a c . Ho r a - sig of t s co c rr t proc ss s ic st 
co-op rat o r a r liabl a co ti all c a gi geo icatio i 
is a p rpl i g a for i abl task. sig fla s ar iffic It to pr ict a for 
t ost part t ct as probl sit sig ill o 1 b co appar t 
t algorit is i pi t at ac o of t t ork. 

t a p r 1 soft ar 1 It I t r t is co pos of a s all i t racti g 

proc ss s ac of ic as a goal t at is cl arl at t 1 1 of t at 

proc ss: t at of co icati g ata to a r c i i g p r proc ss. t a ig r 
1 It coll cti i t ractio of t s proc ss s is a ir ct r s It of t r 1 s 
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t at go r a si gl co icatio .Ho r 1 ss ar car f 1 at t lo r 
1 It rg t a ic at t at ig r 1 1 ill b pr ictabl 1 a i g 

to t ork CO g stio a all collaps . o b car f 1 lop t at t is 
lo r 1 1 ca CO trol t ig r-1 1 rg t a ic. 

lopi g a apti tra sport co trol protocols for co icatio s t- 
orks as larg a ig 1 a ic as t I t r t as i r t a co flicti g 
probl s. pri ar f ctio of r liabl co icati g ata i ol s t co- 
or i atio of r ot proc ss s. is is a iffic It probl ic is co plicat 
f rt r t CO icatio i s ibits pr ictabl loss c ar- 

act ristics. i g to t is iffic It co icatio st b p rfor i a 
fiici tar. protocol st s t t ork r so rc s i a fri 1 a - 
r it r sp ct to p r protocols ic is i co flict it t ai of fiici c . 
It st b abl to tctcagsit t ork iro t a co s i- 
t r or or 1 ss of its r so rc s p i g o t circ sta c s aro it. 
protocol st satisf all of t s a s it o t global i for atio abo 1 1 
stat of t t ork a f rt r still it o t global co trol o r t pat a 
pack t a tak . 

f a tal goal oft is r s arc is to lop a to olog ic ill 
alio t s t sis of a apti protocols a pro i i sig ts i to t sig - 
spac ofs c algorit s. at rt a a protocol sig r a -crafti gt it 

stat ac i at t art of t protocol / s ill sp df t sol tio spac 

a a t ss f ctio ic ill ass ss sol tio s agai st t protocol 

r q ir ts. sig r ill t s tic arc t c iq s to locat 

sol tio s ic t s t t ss f ctio a so t t r q ir ts. 
I ctio 2 so o t ario s co po ts of o r to olog ork 
tog t r to ol orki g protocols. 

of t ost i porta t f ctio s of a co icati g s st is t abil- 

it to CO icat ata r liabl si g a r liabl i . If a s st ca 
b lop i ic r liabl protocols ca b s t sis t it ill go so 
a to pro i g t at ig r-1 lb a io rs ar also attai abl si g t sa 

t c iq . I o r rst p ri t scrib i s ctio 3. s o t at r liabl 

protocols sig iffr ttc iq ssc as ack o 1 g ts a r at 

tra s issio s ar ol r iff r t si latio co itio s. factor 

c a 1 capacit i to o r probl spac c a 1 tilisatio b co s a i por- 
ta t as r of t t ss of t sol tio . I t s co p ri t scrib 
i ctio 3.2 s o t at i o i g protocols ar ol ic a t 

abilit to r t c a 1 at % tilisatio . co cl it a isc ssio 
a gi poi t rs as to o protocols ic a apt to co g stio ca b ol . 

1 

t ork ca b i as a irt al iro t. protocol is a co icat- 
i g tit of t at iro t os actio s ar ictat b its rl i g it 
stat ac i . i c i t is r sp ct a protocol is st a co trol arc it ct r it is 
cl ar t at i or r to ol a apti co icatio protocols ca s t 
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sa ol tio ar t c iq s as t OS s it pro ctio of robot co trol 
arc it ct r s. 

I or r to ol t ork protocols r q ir t i pi tatio of t 
folio i g CO po ts: 

pr s tatio of protocol 
arc strat g 
CO i g sc 

p rators to a ip lat co i g 
t ork si latio 

it ss f ctio 

. pr s t t f pr t c 

r pr s tatio a c os to s is a t sio of t it stat 

ac i ( ) call t co icati g it stat ac i ( ). a 

c os to s t as o r r pr s tatio for t o i porta t r aso s. irstl 

protocols ar asil r pr s t si g t a t is as b a opt b 

t protocol r s arc co it as a sta ar r pr s tatio [6] [4]. co 
t as b s i CO p t r sci c a arti cial i t llig c for a 

ars a its c aract ristics ar 11 oc t a rstoo . s arc rs 

a s t as a r pr s tatio for opti isatio probl s for so ti 

si g bot tra itio al a ol tio ar s arc strat gi s [ ]. 

iff rs fro t i t at it ca aff ct as 11 as r act to 

t iro t i ic it op rat s. pass s ssag s or sig als ia 

bo I q s. ac ac i as a i p t a a o tp t q a 

c t s a t o-st p algorit . ri g t rst st p i p t sig als ar al at 
a a tra sitio r 1 s 1 ct .It s co st p t o tp t sig al is p at 
a t ac i c a g s stat . If o of t r 1 s for a stat ca b c t 
t ac i r ai s i its c rr t stat . 

. rc tr t g 

a tal to t is ork is t abilit to i t protocol as a co trol ar- 

c it ct r ic ca b r pr s t i so s bolic for . is r pr s tatio 
st b o t at ca b tat o r s cc ssi it ratio s t ro g t s arc 

spac of possibl sol tio s. is tra sfor s t probl fro o of co troll r 
lop t i to o of CO troll r t s t . task of g rati g co - 
pi CO trol arc it ct r s si g s arc as b acco plis i ot r Is 
si g tic algorit s( s). ill att pt to co str ct co icatio 
protocols si g si ilar pri cipl s. 

c g c 

CO i g sc s to r pr s t t sol tio i t probl spac is 

clos 1 ti to t s arc strat g plo . ob io s c oic of r pr s tatio 
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si g g tic algorit s is a bi ar stri g i ic ac tra sitio is 

r pr s t b as q c ofbit al s. I a itio to alio i g as a ip latio 

b t sta ar g tic op rators t is r pr s tatio ill also 1 its If asil 
to ariabl 1 gt co i g. bi ar co i g of t is k o as t 

go a r pr s ts a sol tio i t s arc spac . rior to al atio t 

go is tra sfor i to t p o a i sta tiatio of t sol tio i 

sig spac . 



.4 p r t rs t pu teg 

g tic algorit plo ri g t is r s arc s s t sta ar g tic 

op rators i. . tatio crosso r a itio a s btractio . or t sak of 
clarit t folio i g s ctio s tail t act al i pi tatio s . 



ut t . g r is sp ci for ac r r lati g to t 
p r g o . ri g ati g ac bit i t stri g is gi 
of tatio accaebiga qal proportio of t 
go . ac bit i t stri g is t c ck agai st t 
rat a a tatio flips t bit al . 



b r of tatio s 
a i i i al c a c 
tatio rat for t is 
r calc lat tatio 



r ss r. a for s of crosso r ist a ca op rat o aprg orpr 

bit basis, or all t p ri ts carri o t ri g t is r s arc si gl poi t 
crosso r as pro a q at . or t ost part t is as b i pi t o 
aprg basis r t crosso r poi t is r strict to g bo ari s. 

t u tr ct . a itio a s btractio op rators alio 

for a t oro g ploratio of t s arc spac a i g or r o i g g s fro 
t go ra o 1 . itio a s a ra o 1 co str ct g to t 

go . It o g o s 1 ctio pr ss r for si is i tro c b t t ss 

f ctio t r acilogrgo scrats rob st sol tio s it r gar 
tatio . is is b ca s t orki g g s of t sol tio (t parts of t 
it stat ac i t at co trib t to t ss) ca i t s 1 s a o gst t 
r atg soft go 

. t r u t 

or li ar opti isatio probl s ac ca i at sol tio is al at agai st 

a static t ss f ctio ic pro i s a as r of its abilit to sol t 
probl . or o -li ar probl s s c as t os fac r st al at 

a protocol’s f ctio alit i its orki g iro t. or a applicatio s 
t is is i practical to t a o t of ti r q ir for a al atio a a 
CO p t r o 1 st b s to si lat t ag t’s iro t a its i t r- 
actio it it. is is ot t cas for t applicatio propos ; al atio of 
a protocols p rfor a c co 1 tak plac i a r al t ork it o t r aso abl 
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ti ora alt o g at t is stag r al orl al atio o 1 a c- 

ssar CO pi it . at is r q ir t is a t st b i ic to al at t 
p rfor a c of a protocol it i its orki g iro t or as clos to t at 
iro t as is possibl . 

folio i g p ri ts s a si pi s r-r c i r t ork si latio 
a a pi of ic ca b s i g r . o i itiat a si latio top - 
o s ar CO str ct : a s r a a r c i r. co po t of ac 

p o is g rat fro t go to b al at . ot s r a r c i r 
a a associat co icatio c a It c aract ristics of ic ar 
p tot pri tbigp rfor . picall ac p o also as 
a tra s issio bffra arci bffr. t s ig aci t tra s- 
issio b ff r CO tai s t ssag to tra s it. I a itio ac ac i also 
as so for of c rr t or i ic ata it s ca b It poraril 

b for big plac i to b ff rs or ssag s. 

si latio c t s for a sp ci c b r of ti st ps. ri g ac 

stpt s ra t t rcirp o sgtacacto ct. rig 
c tio ac ac i it rat s t ro g t tra sitio s of its c rr t stat fro 

rst to last. actio associat it t tra sitio is al at if t actio 
ca b p rfor t tra sitio r s. I ff ct ac tra sitio for s a bool a 
pr ssio . If t pr ssio ol s t t ac i ak s t tra sitio to t 
t stat . t ac ti st p i t si latio r possibl tra sitio of t 

c rr t stat is att pt til a c tio is capabl of ri g. t t is poi t 
t ac i c a g s stat to t stat i icat b t t-stat 1 of t 
tra sitio . 

t r ork ill i ol larg r si latio si ic a si lat t ork 

o s ill CO icat across a si gl s ar c a 1 it ac o plo - 

i g t sa protocol as i t r al orl . al atio ill b a as r of 

t global traffic t ro g -p t across t c a 1. o ti ork a s t 

c rr t r al- orl I t r t i plac of t si latio . 

.6 t ss u ct 

p rpos oft t ss f ctio is to al at t p rfor a c of t protocol 

ri g its c tio i t t ork si latio a pro i so q a ti abl 
as r of its f ctio alit . as r its If s o 1 i cl as f co po ts 

as possibl si c t gr at r t b r of co po ts t 1 ss pr ictabl t 

pop latio ’s CO rg c o t sir poi t i sol tio spac . is as r 

ca b calc lat i t o a s: t ost iffic It is to gat r i for atio ri g 

t si latio a calc lat t t ss fro t ata coll ct ; a asi r optio 
is to as r t t ss of t stat of t t ork o s aft r t si latio 

as is 

st b ar i i t at ass ssi g t al of a sol tio si g o 1 a 

si gl si latio os ot pro c a acc rat as r of its abilit si c it 

is possibl t at t protocol is ploiti g so asp ct of t t st ata s . I 

or r to gai a fair as r of t t ss of a sol tio ac st b t st a 
b r of ti s si g iff r 1 1 st ata. st b car f 1 s 1 cti g t 
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b r of r -t sts to p rfor if s too a r -t sts t p rfor a c ill 
gra a if s too f t sol tio ill ot b a q at 1 t st . is 
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CO icatio protocol ca b g rat si g ol tio ar s arc . o pro i 
a as r of t q alit of t s t sis protocol ac ca i at sol tio 

as al at si g a t ork si latio ic lat s t iro t for 
ic t alt r ati g bit protocol is a opti al sol tio . It as op t at t 

s st o 1 pro c a ac i it si ilar stat tra sitio s a t r al b - 
a io r. alt r ati g bit protocol is a si pi protocol t at pro i s r liabl 
CO icatio si g a alt r ati g ali atio bit it i icat s t r or ot 
a ssag as lost ri g tra s issio . It is 11 oc t it lit rat r 
a as rst propos i [ ]. 



t r u t . o g rat as r si ilar to t at of o r b c ark 

ac i t si latio st clos 1 o 1 t op rati g iro t of t 
origi al protocol, or t is t c a 1 st a t capacit to ol o 1 a 
si gl ssag . I accor ac itt rfrc protocol; o 1 ssag s ca b 
lost a if a ssag is lost tea 1 ill flip t bit i t ssag a r. 
or a sc atic o r i of t si latio s g r . 

ct s. folio i g actio sab trapolat fro t ac i sp c- 
i i t origi al alt r ati g bit protocol. 

: a ssag o t co icatio sea 1 taki g ata fro 

or if r q ir 

c i : c i a ssag fro t co icatio sea la plac t 

ata i to or . 

q : lac t it i or i to t r c i b ff r. 

q : lac t rst 1 t of t tra s it b ff r i to or . 
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t rk si ulati n f r luti n f r liabl c unicating finit stat achin . 



11 : 11 actio al a s orks 

ac of t s actio s r t r s a tr or fals bool a al p i g o its 
abilit to CO pi t t sp ci task. 1 for t s actio is t is ot t 
cas it ill r t r tr r all co itio s to pr t it tio al f back 

fro t r c i i g ac i .1 t cas r t c a 1 air a co tai s a 

ssag t ssag is iscar 



t ss u ct . t ss for a si gl protocol is bas o t proportio 

of a ssag r c i aft r a b r of ti st ps. o s r a r liabl 

protocol is pro c o 1 t i -or r portio oft r c i ssag is co si - 

r . g o ill scor ro if all b t t rst ssag as s cc ssf 11 
r c i . If t is as ot t cas a t tir proportio of t ssag as 
CO si r t t pop latio o 1 pr at r 1 co rg o a r liabl 

sol tio a b CO i capabl of back-tracki g t ro g t s arc spac to 
r liabl sol tio s. 



su ts. so i g r 3 as b g rat ri g a t pical r 

of t s st . s it a t pical stat tra sitio iagra ac o r pr s ts 

a stat . ac tra sitio is r pr s t as a arc fro t stat i ic it is 
c t to t t t stat . tra sitio is ark it t actio to b tak 
a t para t rs for t at actio . o actio s r q ir o para t rs i ic 

cas t is r at co i g i t tra sitio a s tralit to t go : 

i cr asi g its rob st ss it r gar to tatio . tra sitio s ar ark 
it t or r i ic t ar to b att pt so i sq ar brack ts. 
s ca b s it iagra t s r co ti all s s a ssag it 

i ti r “ ” . 1 aft r r c i i g a ack o 1 g t fro t r c i r o s it 

q t t ata it . is is a si pi stop-go strat g a is t clos st 

a CO to g rati g t alt r ati g bit protocol, ot : ac tra sitio 

as t sa b r of para t rs alt o g for q q a 11 

t sarsprflosa iscar ri g c tio . 
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s i g Itipl pack ts. ti sp t stori g a pack tit r tra s issio 
b ft r as s to s a s co pack t s cc ssi pack ts a i g far 1 ss c a c 
of loss tat pr io s. o fore t lop t of a r tra s issio sc 

ac tra sitio as b assig a para t r ic sp ci s t 1 gt of ti 
it a tak to c t t actio . 
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Abstract. Modern VLSI design methodologies and manufacturing technologies 
are making circuits increasingly fast. The quest for higher circuit performance 
and integration density stems from fields such as the telecommunication one 
where high speed and capability of dealing with large data sets is mandatory. 
The design of high-speed circuits is a challenging task, and can be carried out 
only if designers can exploit suitable CAD tools. Among the several aspects of 
high-speed circuit design, controlling power consumption is today a major issue 
for ensuring that circuits can operate at full speed without damages. In 
particular, tools for fast and accurate estimation of power consumption of high- 
speed circuits are required. In this paper we focus on the problem of predicting 
the maximum power consumption of sequential circuits. We formulate the 
problem as a constrained optimization problem, and solve it resorting to an 
evolutionary algorithm. Moreover, we empirically assess the effectiveness of 
our problem formulation with respect to the classical unconstrained 
formulation. Finally, we report experimental results assessing the effectiveness 
of the prototypical tool we implemented. 



1. Introduction 

Modern telecommunication systems must ensure high bandwidth and high 
performance to cope with the increasing amount of delivered data. Different solutions 
at different levels of abstraction can be exploited: improved communication channels, 
better communication protocol and high-speed communication equipment. Modern 
VLSI design methodologies and manufacturing technologies are making circuits 
every day faster. To cope with the complexity of designing high-speed circuits 
designers must exploit suitable CAD tools. Among the several aspects of high-speed 
circuit design, power consumption is today a major issue. In the last years, design for 
low-power consumption has became a widespread design paradigm. Design 
techniques to control power consumption are mandatory because excessive power 
dissipation can cause performance degradation, run-time errors, or device destruction 
due to overheating. Large instantaneous power dissipation may also cause local hot 
spots that have negative impact on circuit reliability. With increasing demands for 
reliable high-speed circuits accurate estimation of peak-power dissipation during 
design process is becoming essential. 
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In this paper we address the peak-power estimation problem for high-speed 
sequential circuits. Power consumption in sequential circuits is a non-linear function, 
Pf , of three variables: the circuit initial state, S, and two vectors ( , V , ) that are 

applied to the circuit inputs at time t-1 and t. We represent them as a 3-tuple (S, , 

V , ). The problem of computing the peak-power consumption of sequential circuits 
can be formulated as the problem of finding the 3-tuple {S, , F, ) that maximizes 

the function Pf . 

Several techniques have been proposed to solve this problem resorting to either 
exact approaches [1] or techniques based on Automatic Test Pattern Generators [2]. 
But only in [3] and [4] an algorithm is proposed able to cope with large sequential 
circuits. In particular the peak-power problem is formulated as an unconstrained 
optimization problem and a technique based on Genetic Algorithms is proposed. This 
technique is strongly approximated since the reachability property of the state S is 
neglected during the optimization process. As a consequence, this approach could 
lead to solutions where the 3-tuple (5, F,_i , F, ) embeds a state S that the sequential 
circuit never reaches. To make the 3-tuple a feasible solution, the authors of [3] and 
[4] propose to replace at the end of the optimization algorithm the state S with a 
reachable one as close as possible to S. 

In the following, we will analyze the behavior of this algorithm, and 
experimentally show that it can underestimate peak-power when the number of 
reachable states is a small fraction of all the possible states. We will then propose an 
approach that overcomes this drawback by exploiting the knowledge about the 
reachable state set during the optimization process. In this paper we propose and 
evaluate an implementation of our peak-power estimation algorithm that is based on 
the Selfish Gene algorithm [5], an evolutionary optimization algorithm we developed. 
We support our approach by reporting experimental results assessing the effectiveness 
of our approach. 

The algorithm presented here is based on a previous work [8] where Genetic 
Algorithms were used to solve the peak-power problem. In [8] a Genetic Algorithm 
evolves a population where each individual is a 3-tuple composed of a reachable state 
and two vectors. To guarantee that the population contains only valid individuals, 
each time crossover and mutation operators are applied the state in the new individual 
is analyzed, and replaced with a reachable one when required. This approach can limit 
the search capability of Genetic Algorithms and thus the effectiveness of our peak- 
power estimation algorithm. By resorting to the peculiarities of Selfish Gene, state 
modification is no longer required, and thus the effectiveness of our algorithm can be 
improved. 

The remainder of the paper is organized as follows. Section 2 presents a formal 
description of the problem. Section 3 describes the evolutionary algorithm we exploit 
in our tool, and Section 4 presents the peak-power estimation algorithm we 
developed. Section 5 reports experimental results on some benchmark circuits. 
Finally, Section 6 draws some conclusions. 
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2. Problem formulation 

We assume that the sequential circuits we are analyzing are manufactured with 
CMOS technology. Power consumption in CMOS circuits is mainly due to the 
number of times the output of circuit gates switches from 0 to 1 and vice-versa. Peak- 
power consumption is defined as the maximum power consumption that takes place 
when a couple of input vectors are applied to the circuit inputs, starting from a given 
initial state. The procedure to measure peak-power can be outlined as follows: 

1. the circuit memory elements are initialized to a given state S; 

2. an input vector is applied to the circuit, and a clock pulse is applied; 

3. an input vector V, is applied to the circuit, a clock pulse is applied and the power 
consumption is measured during the transient period. 

The first two steps lead the circuit to a configuration where the circuit nodes hold 
known values (either 0 or 1). We refer to the power measured during the last step as 
Pf , and use to notation Pf (5, , T, ) to indicate that power is function of the 3-tuple 

{S, V,_i , V, ). The classical peak-power problem formulation that can be found in [3] 
and [4] is the following: 

where PP is the peak-power consumption, B = {0,1} , n is the number of memory 
elements in the circuit and m is the number of circuit inputs. This formulation neglects 

the fact that in sequential circuits the number of reachable states is often less than 2" . 
Therefore, algorithms that solve equation (1) could compute 3-tuples that correspond 
to PP values that cannot be obtained, since the memory elements can never reach the 
configuration S during circuit operations. Power figures attained by solving equation 
(1) are overestimation of actual peak-power consumption, and if adopted during 
design could lead to unnecessary expensive design solutions. The authors of [3] and 
[4] suggest replacing an unreachable state in the solution of (1) with the closest 
reachable one. This approach weakens the correlation between S and {S, , F, ) and 

usually lead to peak-power figures that underestimate the actual power consumption. 
As a consequence, the obtained peak-power prediction could lead to wrong design 
solutions. 

The problem formulation we propose is the following: 

PP = M4X(P^(5,F,_i,F,)),5eZ,FeB'" (2) 

where Z c B" is the set of states that the sequential circuit under analysis can reach 
starting from the reset state, i.e., the state where all the memory elements are set to 0. 
By solving equation (2) and therefore by considering reachability of the state 5 as a 
dimension of the search space, we compute peak-power figures more accurate than 
what equation (1) provides. 
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3. The Selfish Gene algorithm 



The Selfish Gene algorithm (SG) is an evolutionary optimization algorithm based 
on a recent interpretation of the Darwinian theory. It evolves a population of 
individuals seeking for the fittest one. In the selfish gene biological theory, population 
itself can be simply seen as a pool of genes where the number of individuals, and their 
specific identity, are not of interest. Therefore, differently from other evolutionary 
algorithms, the SG resorts to a statistical characterization of the population, by 
representing and evolving some statistical parameters only. Evolution proceeds in 
discrete steps: individuals are extracted from the population, collated in tournaments 
and winner offspring is allowed to spread back into the population. 

An individual is identified by the list of its genes. The whole list of genes is called 
genome and a position in the genome is termed locus. Each locus can be occupied by 
different genes. All these candidates are called the gene alleles. In the context of an 
optimization problem, looking for the fittest individual corresponds to determine the 
best set of genes according to the function to be optimized. 

Since the SG “virtual” population is unlimited, individuals can be considered to be 
unique, but some genes would certainly be more frequent than others might. At the 
end of the evolution process, the. frequency of a gene measures its success against its 
alleles. However, at the beginning of the evolution process, the frequency can be 
regarded as the gene desirability. When the majority of a population is characterized 
by the presence of a certain characteristic, new traits must harmonize with it in order 
to spread. 



genome SG (VirtualPopulation P) 

{ 

genome BEST, Gi , G2 , winner, loser; 

iter = 0 ; 

BEST = select_individual (P) ; // best so far 

do { 

++iter ; 

Gi = select_individual (P) ; 

G2 = select_individual (P) ; 

tournament (Gi , G2); // identify winner and loser 
increase_allele_f requencies (P, winner) ; 
decrease_allele_f requencies (P, loser) ; 
if (winner is preferable to BEST) 

BEST = winner; 

} while (steady_state 0 ==FALSE && iter<max_iter) ; 
return BEST; 

} 



Fig. 1 : Selfish Gene algorithm pseudo-code. 

The pseudo-code of the SG algorithm core is reported in Eig. 1. Eurther 
implementation details about the SG algorithm are available in [9], while biological 
motivations are better analyzed in [10]. An extension to the basic SG algorithm to 
deal with more complex fitness landscapes is described in [11]. 
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4. Power estimation algorithm 

The peak-power estimation algorithm we developed is composed of the following 
steps: 

1. the set of the reachable states Z c B" is computed, which cardinality is |z| . It can 

be computed either resorting to exact symbolic calculation techniques [7] or 
through logic simulation; 

2. peak-power estimation is performed. The SG is run to solve equation (2) where the 

genome represents the 3-tuple (S, , F, ) and is composed of 1 + 2 -m loci. The 

first locus is an index ranging from 0 to |s|-l representing a reachable state 
5 e S . The remaining loci are binary values, coding the couple of vectors ( F,_i , 

During step 2, the power consumption /’^(5,F,_[,F, ) is computed resorting to a 

unit-delay logic simulator. The adoption of a logic simulator is a well-known effective 
approach to measure power consumption in CMOS circuits [6], since it conjugates 
simulation speed with accuracy. 



5. Experimental results 

A prototypical version of our algorithm named SG-ALPS, Selfish Gene-based 
AnaLyzer of Power in Sequential circuits, has been written, which implements the 
above-introduced procedures. The tool consists of about 500 lines of ANSI C code 
and exploits the SG and logic simulation packages developed at our institution. 
Reachability analysis has been performed resorting to exact calculation techniques 
exploiting the BDD [7] package developed at our institution. 

The subset of ISCAS’89 sequential circuits tractable with symbolic calculation 
techniques have been used to evaluate the performance of SG-ALPS: all the 
experiments have been performed on a Sun UltraSparc 5/333 with 256 MB RAM. To 
compare SG-ALPS with a state-of-the-art tool, we have re-implemented the algorithm 
proposed in [4]. Experimental results of our re-implementation of [4] are the same of 
[4]; we can therefore perform a fair comparison between the two tools. 

Two sets of experiments have been performed. The first one aims at empirically 
showing that solving equation (1) as done by [4] could greatly underestimate peak- 
power consumption. Conversely, the second set of experiments aims at assessing the 
effectiveness of the approach we propose. 

Table 1 reports results we gathered with our implementation of [4]. The first 
column reports the benchmark name, PPU and PPR report respectively the peak- 
power consumption obtained by solving equation (1) (which takes into account also 
unreachable states) and the power obtained by replacing the state in the solution with 
the closest reachable one. Column A reports the difference between these power 
figures, while column CPU reports the time requirements. 

By observing the column A, one can observe that several circuits exist where peak- 
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power strongly depends on the reachability of the initial state S and on the correlation 
between S and the vectors ( , F, ). Where a significant loss in peak-power is 
found, a large portion of the state S has been modified to make it reachable, thus a 
high difference exists between the ideal initial state and the selected reachable one. 
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PPU 
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A 
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0.855 
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0.927 
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-6.40 


22.7 


sl494 


1.234 
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-6.39 
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Avg. 






-15.96 


10.4 



Table 1. Analysis of algorithm [4] 



Table 2 reports results obtained by running SG-ALPS when max_iter (Fig 1) is 
set to 1000. Column SG-ALPS reports the peak-power figures predicted by our 
algorithm, while the third column reports the best results attained by the algorithm 
proposed in [4]. We compare the two algorithms in column A. Finally, the CPU time 
requirements are reported. 

As far as peak-power estimation accuracy is concerned, we can conclude that our 
approach is superior to [4]. Even if on the average SG-ALPS attains PP figures 21% 
higher than [4], several circuits exist where SG-ALPS computes power figures 50% 
higher than [4], thus showing the importance of considering reachability during the 
optimization phase. 

As far as CPU time is concerned, the algorithm proposed in [4] is far more 
effective than SG-ALPS. This is mainly due to two factors: SG requires more time to 
converge than Genetic Algorithms, and guaranteeing state reachability during the 
optimization process is a time consuming operation. 
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We do point out that the CPU requirements of SG-ALPS, although higher than 
those of previously proposed approaches, still remain in the order of minutes, which is 
negligible with respect to the time required by most steps in the current circuit design 
flow. The increase in the CPU time is worth paid by the improvement in the attained 
estimation quality. 

When compared with the method proposed in [8], implementing a similar approach 
using Genetic Algorithm, we observed that the algorithm proposed here provides 
comparable results. However, the adoption of the SG algorithm allows to more easily 
deal with optimization problems for which a heterogeneous solution encoding is more 
suitable. 
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Table 2. SG-ALPS results 



6. Conclusions 

High-speed telecommunication equipment is required to cope with the market 
request for high-speed data delivering. When designing high-speed circuits for 
communication equipment, particular care must be posed to the control of power 
consumption. In particular, peak-power estimation is mandatory for designing today 
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high-speed circuits. In this paper the peak-power prediction problem has been 
formulated as a constrained optimization problem, and an algorithm based on a new 
evolutionary paradigm has been proposed. The algorithm is designed for addressing 
sequential circuits and thanks to its ability of guaranteeing feasibility of solutions 
during the optimization process it is more effective than other approaches. 

Moreover, in this work we experimented that SG is better suited to address 
optimization problems having heterogeneous solutions, i.e., solutions mixing 
components defined over different domains. As an example, in peak-power estimation 
the solution embeds a state that is naturally expressed as an index in a state table, and 
a couple of vectors, represented as a collection of binary values. 

We are currently working toward an implementation of SG-ALPS where 
approximated reachability analysis is performed through logic simulation. Thanks to 
this improvement, we will be able to address large sequential circuit currently not 
tractable by symbolic calculation techniques. 
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Abstract. This paper presents a simple communication architecture 
for Multi- Agent Learning Systems. The service provided by the commu- 
nication architecture allows each agent to connect to the user interface, 
the application and the other agents. The communication architecture is 
implemented using TCP /IP. An application example in a simplified traf- 
fic environment shows that the communication architecture can provide 
reliable and efficient communication services for Multi-Agent Learning 
Systems. 

1 Introduction 

Many researchers in the field of Distributed Artificial Intelligence are beginning 
to build agents that can work in a complex, dynamic multi-agent domains [1]. 
Such domains include virtual theater [2] , realistic virtual training environments 
[1], RoboCup robotic and virtual soccer [3] and robotic collaboration by obser- 
vation [4]. This is because that there is a realisation of the benefits of using 
problem-solving models based upon an interacting group of agents rather than 
a single agent and multi-agent systems can benefit from the inherent properties 
of distributed systems, i.e. parallelism, robustness, scalability. 

Learning in multi-agent systems has been seen as important both in re- 
moving the need to “hard code” the agent behaviour, as for certain problems 
the appropriate behaviour is unknown [5, 6]. It is motivated by the insight that 
it is impossible to determine a-priori the complete knowledge that must exist 
within each component of a distributed, heterogeneous system in order to allow 
satisfactory performance of that system. Especially if we want to exploit the 
potential of modularity, such that it is possible for individual agents to join and 
leave the multi- agent system, there is a constant need for the acquisition of new 
and the adaptation of already existing knowledge, i.e., for learning. 

Within this setting, different kinds of learning tasks must be investigated, 
such as ‘traditional’ single agent learning tasks, learning in teams, learning to 
act within a team, and learning to cooperate with other agents. To solve any 
of these tasks, communication, i.e., the existence of appropriate information 
that can be communicated to the learning agents is of primary importance. 



S. Cagnoni et al. (Eds.): EvoWorkshops 2000, LNCS 1803, pp. 255-266, 2000. 
© Springer-Verlag Berlin Heidelberg 2000 




256 N. Ireson et al. 



In this work, we consider the communication issues in the cooperative 
multi-agent learning systems. In a cooperative multi-agent system, the agents 
can cooperate either implicitly or explicitly. With implicit cooperation agents 
act selfishly to satisfy their individual goals but their actions can have beneficial 
effects upon other agents. With explicit cooperation, agents share information 
and rewards, thus perform actions which provide mutual benefit, involving some 
form of direct communication. 

Sen et al showed that agents attempting to optimise the use of limited 
resources converge to optimal states faster with a limited view [7]. They also 
investigated the formation of agent coalitions, where agents within a coalition 
share information about their resource utilisation intentions. The use of such 
coalitions further improves the system’s convergence to the optima. Prasad 
and Lesser examined an explicitly cooperative directly communicating system 
in which agents learning to communicate only relevant information dependant 
on situation specifics [8]. The learning algorithm uses the Instance-Based learn- 
ing paradigm. At the end of a particular problem solving run, agents assess their 
coordination strategy according to four performance measures. Agents then de- 
rive new strategies and broadcast these to the other agents. Seredynski et al 
developed a coevolutionary multi-agent system showing that global behaviour 
evolves via only local cooperation between agents acting without global infor- 
mation about the system. Seredynski introduces an exchange process into the 
game, which redistributes the payoff [9, 10]. The players are placed in a ring 
and play a set number of games with a selected number of neighbours, i.e. local 
interactions. Three exchange schemes were tested: 

1. no payoff exchange, no cooperation; 

2. payoff is exchanged amongst interacting players, local cooperation sharing; 

3. payoff is exchanged amongst all players, global cooperative sharing. 

They considered two simple coevolutionary schemes: loosely coupled GA 
and loosely coupled classifier systems. The term “loosely coupled” refers to the 
individuals in a population being evaluated only on their local rather than a 
global fitness function. The experiments showed that with no payoff exchange 
the players evolve to the Nash equilibrium (both defecting), whilst with both the 
local and global payoff exchange players evolved to cooperate. In fact the payoff 
exchange alters the game’s payoff matrix and encourage cooperation. Seredynski 
applied the technique of payoff exchange to a dynamic mapping and scheduling 
problem [9]. Bull et al applied the learning classifier system design to a multi- 
agent environment in which each of a number of classifier systems represents a 
co-operating communicating agent [11]. It is demonstrated that the evolution 
of multiple co-operating agents can give improved performance over an equiva- 
lent single agent model. The performances of the varying components, such as 
reinforcements, discovery system, in a multi- agent environment, were examined 
in detail in [12]. 

In this work, we extend the framework of the multi-agent systems in [10, 12] 
to a general version and aim at developing a communication architecture for 
these multi-agent learning systems. In these environments, each agent is defined 
by a classifier system which evolves a set of control rules and is concerned with 
agents as software rather than theoretical constructs. In this communication 
architecture, each agent is allowed to connect to user interface, the application 
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Figure 1: Structure of the agent 



and the other agents. An application example is given to show that the commu- 
nication architecture can provide reliable and efficient communication services. 



2 The Framework of the Multi- Agent Learning 
System 

Multi-agent learning systems usually consist of a number of collaborative agents 
and consider how these agents can interact to effectively cooperate in problem 
solving tasks. They have many applications in process control, network man- 
agement, scheduling, etc. [9, 13, 17]. 

As with a number of contemporary fields on computer science, such as AI 
and Artificial Life (ALife) the definition of an agent ranges from a strong to a 
weak notion. On the stronger end of the scale an agent is deemed to possess 
properties akin to those found in humans, such as knowledge, belief, intention, 
etc. The weaker notion of an agent tends to be more pragmatic and associated 
with agent software engineering. Such agents can be defined as possessing less 
anthropomorphic properties, for example [18]: 

• autonomy: operate without the direct intervention of others, and have 
some control over actions and internal states 

• communication: interactions with other agents 

• reactivity: perception of their environment and timely response to changes 
that occur in it. 

• Pro-activeness: not merely reacting to events but exhibit pre-active goal- 
directed behaviours. 

The multi-agent learning system considered in this paper contains a num- 
ber of distributed, communicating agents, where each agent, as shown in Figure 
1, has a learning classifier system (providing the rule base and control actions) 
and a communication server which is used to connect the agent to the user inter- 
face, the application and to other agents. These two elements of the agent are 
separate since; as messages are passed around the agent network, the commu- 
nication server acts independently of the classifier system to route the message 
to its neighbours. Another reason for keeping the communication server distinct 
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Application Agent 



Figure 2: Interaction structure of the agent 



from the classifier system is that communication is likely to be implementation 
specific (even in the test applications) . Thus it is necessary when specifying the 
communication server to consider the general requirements of setting up and 
maintaining the communication in a multi-agent learning system rather than 
those in a specific software and hardware implementation. 

The communication server must provide the following services: 

• Interaction between the application and the agents: The interface between 
the agents and the application is defined by the agents’ detectors and 
effectors. The application must package the system state into a message 
which is in the representation expected by the agents. The application 
receives an action from the detectors which is interpreted to effect the 
system. 

• Interaction between the User Interface and the agents: The amount of 
performance information reported by the agents is parameterised. During 
the development of an application it is likely that the user will wish to 
monitor performance more closely to ensure the system has been correctly 
configured. Another consideration is whether information (performance 
statistics, warnings, error, etc.) is logged to a file or sent to the user 
interface directly. 

• Inter-agent communication: The communication strategy between the agents 
is effected by the application and the learning strategy to be employed. 
Each agent can communicate with a specific other agent or group of agents, 
known as an agents neighbourhood, the user specifies the agents contained 
in each agents neighbourhood. 

The Figure 2 shows a representation of the interactions between an agent and 
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Agent Network 






Figure 3: Structure of the agent network 



its environment which contains three types of actors: the user, the application, 
and other agents. 

While it is important to examine the research into agent languages (and 
implementations) such as KQML, such work provides functionality beyond the 
needs of these systems. The communication in such learning systems will have a 
limited syntax and have a well defined content, thus the language required will 
be fairly simplistic. We have specified the nature of the communication, which 
includes: utility measure (enabling “fitness” sharing); environmental state; ac- 
tions performed or to perform (intentions); shared classifiers. 



3 The Proposed Communication Architecture 

3.1 The Whole Agent Network 

An agent network is defined by the characteristics of the application, which de- 
termines the appropriate distribution and connections between agents. Each 
agent is allocated a specific address, and a port for each channel of communica- 
tion, this includes communication with the user interface, application and other 
agents. 

There are two basic protocols of communication. The simplest is to broad- 
cast the message which is received by all the agents, the message can include a 
tag to identify the sender and nature of the message. It is the responsibility of 
the receiving agent to determine whether to utilise or ignore the information in 
the message. The second means of communication is to send messages to the 
agents in the local neighbourhood. In this approach each agent is the centre of 
a neighbourhood, those agents contained in the neighbourhood (from none, i.e. 
no communication, to all other agents) receive any messages sent by the central 
agent. As neighbourhoods might overlap an agent can be a member of a num- 
ber of neighbourhoods, for example, for the agent network, shown in Figure 3, 
neighbourhoods are constructed from an agents nearest neighbours, thus sixteen 
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groups, each centred on one agent, are formed: 

Group 1: 1, 2, 5 

Group 2: 2, 1, 3, 6 

Group 3: 3, 2, 4, 7 

Group 6: 6, 2, 5, 7, 10 

Group 7: 7, 3, 6, 8, 11 

Group 16: 16, 12, 15. 

This approach is general known as multicast or one-to-many communica- 
tion. There are a number of possible approaches to this form of communication 
in distributed systems from flooding where each node sends a copy of the incom- 
ing message to all the connected nodes (except for the message source node) to 
routing where messages follow a pre-specified path to their destination node. 

The principal problem with flooding is that it causes a great deal of re- 
dundant communication, also as each node must check if it has already received 
each incoming message, with frequent communication this can lead to a commu- 
nication overload. Routing requires a more complex initialisation process but 
minimises the communication traffic. Each node when it receives a message 
refers to a lookup table giving the nodes to which copies of the message are sent. 
Unlike flood broadcast there is no redundant communication, thus the severing 
of a communication link will cause at least one node to fail to receive messages. 
In practice the choice of method should reflect the constraints of the system, i.e. 
the trade-off between fault tolerance and communication load. 

3.2 Initialisation of the Agent Network 

The initialisation process firstly involves the setup and opening of communication 
channels from; agent to user interface, agent to application and agent to other 
agents. Once this has been successfully completed, the classifier system can be 
initialised, the agent then waits for the first message from the application to 
begin its control process. 

The initialisation of communication involve each agent connecting to the 
user interface, application and other agents. All these channels might involve 
two-way communication. During the initialisation the agents open a communi- 
cation channel and await a connection message. The channel is tested to ensure 
the communication is setup correctly as although the configuration parameters 
have been previously checked for consistency, the parameters may be inconsistent 
with the physical communication process, also this process might be faulty. 

Although the term “socket” is used in the specification as the medium 
to connect communication channels in implementation other methods can be 
used, such as calls to remote objects, when using RMI or DCOM. The basis of 
the communication initialisation and run-time processes are not affected. The 
creation of the communication object and binding in a remote registry (on a 
given hostname and port) replaces the creation of a server socket and calls to 
the remote object replace read and write calls to the sockets. 

Note that it is possible for the communication server to create separate 
processes to listen on the communication channel for messages, this allows the 
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Figure 4: Structure of communication server 



agent to be reactive to external messages. The communication with neighbours 
requires a single channel for incoming messages, and separate channels from 
sending to each neighbour (except if the messages are broadcast on sent via a 
proxy). The initialisation of the communication server, as shown in Figure 4, 
involves the following steps: 

1. The Communication Server object creates the specific Communication Ser- 
vices (Application, User Interface or Neighbourhood) as specified by the 
configuration. 

2. The Communication Server object create a monitor which maintains the 
list of current connections. 

3. The Communication Server object passes the Communication Service ob- 
ject and connection configuration information to the Connector object 
which, for connection with the User Interface and Application and incom- 
ing channel from the neighbouring agents, opens a Server Socket on the 
specified port and waits for a request to connect. For the outgoing channel 
to the neighbouring agents the Connector object intermittently requests a 
connection to the neighbours specified port. 

4. The User Interface, Application or Neighbouring Agent sends a request to 
connect. 

5. The request to connect is accepted by the neighbour’s server socket. 

6. The Connector sends the Communication Service object and open socket 
to the Communication Monitor. 

7. The Communication Monitor object tests the communication channel, if 
the test succeeds the Communication Service is passed to the Connection 
object, otherwise the socket is closed and the failure reported. 
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Figure 5: Class diagram of the communication 



8. The Connection object starts the thread to handle the connection and 
passes the input and output streams to the Communication Service object. 

3.3 Logical View of the Communication 

A class diagram of the communication is given in Figure 5. The function of each 
class (object) in the communication is described briefly as follows: 

• Communication Server: The Communication server supports a number of 
multi-threaded channels. It creates (or opens) each channel on a specified 
port allowing the Agent to communicate with the User Interface, Appli- 
cation and other Agents. It provides the ability to send messages whilst 
listening on the port for incoming messages which are passed to the ap- 
propriate objects. 

• Connection Monitor: The Connection Monitor object maintains a list of 
the current connections. The thread waits to be notified if a connection 
terminates and updates the list. 

• Connector: The Connector class either listens for a connection on a spec- 
ified port (using a server socket) or connects to another agent’s server 
socket. Once accepted the socket is sent to the Connection Monitor. 

• Connection: Connection objects are created by the Connector thread using 
the Communication Monitor method addConnectionQ . It simply creates 
a thread to handle the connection. 
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Figure 6: The simulated traffic environment 



• Communication Service: A general class for each of the types of commu- 
nication required. 

• Listener: An object that listens on a communication channel for an incom- 
ing message. 

• User Interface: The communication service require by the User Interface. 
This object handles messages received from or sent to the User Interface. 
Incoming messages are interpreted and, if necessary, call the appropriate 
function. Outgoing messages are packaged and sent to the User Interface. 

• Application: The communication service require by the Application. This 
object handles messages received from or sent to the Application. Incom- 
ing messages (generally system state or rewards) are interpreted and, if 
necessary, sent to the appropriate objects (such as classifiers) in the clas- 
sifier system. Outgoing messages (general actions) are packaged and sent 
to the Application. 

• Neighbourhood: The communication service require by the Agent Neigh- 
bourhood. This object handles messages received from or sent to the Agent 
Neighbourhood. Incoming messages are interpreted and, if necessary, sent 
to the appropriate functions in the classifier system. Outgoing messages 
are packaged and sent to the neighbouring agents. 

4 Application Example 

Optimization of a group of traffic signals over an area is typical multi-agent type 
real-time planning problem without precise reference model given. To do this 
planning, each signal should learn not only to acquire its control plans individu- 
ally through reinforcement learning but also to cooperate with each other. This 
requires communication between the agents. In this example, we developed a 
multi-agent learning system, which is aimed at learning the efficient control rules 
for the dynamic traffic environment and with the communication provided by 
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Figure 7: Performance comparison of different control strategies 



the developed communication architecture. Each agent has a classifier system 
(providing the control strategy) and a communication server which is used to 
connect the agent to the user interface, the application and to other agents. 

To control a traffic network, we associate an agent to each junction of the 
traffic network. The agents are initialised according to the traffic network con- 
figuration and user-specified parameters. For the simulated 2x2 traffic network, 
as shown in Figure 6, four agents, i.e., agents I, II, III, and IV, associating with 
junctions I, II, III, and IV, are need to provide comprehensive control of the 
network. Agent I has the neighbouring agents II and III, and agent II has the 
neighbouring agents I and IV, etc. The communication server in each agent 
provides the control actions of its neighbouring agents, and these information is 
used to construct control rules for its junction. 

The classifier system employed is a version of Wilson’s “zeroth-level” system 
(ZCS) [19], with some changes on the classifier representation [20]. The condition 
part of each classifier consists of six bits, which reflects the scalar level of queue 
length from each direction and the previous actions of the neighbouring agents. 
In this application, the scalar level of the queue length is set to 4, which ranges 
from 0 to 3, corresponding to the four linguistic variables, {zero, small, medium, 
large }. The action part indicates the required state of the signal. For instance, 
for junction I, the rule 130201:1 says that if the queue from directions east and 
west are small (1) and zero (0), but the queue from directions south and north are 
large (3) and medium (2), and the previous neighbourhood junction controllers’ 
actions are vertically red (0) (junction II) and green (1) (junction III), then the 
traffic light stays green vertically (1) for a fixed period of time. The performance 
evaluation, reinforcement learning strategy, genetic algorithm and the simulated 
traffic environment are all similar to those used in [20] . 

For comparison purpose, two types of control strategies are employed: ran- 
dom control strategy and the developed multi-agent learning system (MATS) 
strategy. The random control strategy determines the traffic light’s state ( 0 
or 1) randomly at 50% of probability; whilst MATS strategy determines the 
traffic light’s state according to the action of the winning classifier of the agent. 
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Experiments were carried out for three different types of traffic conditions. In 
these simulations, the mean arrival rates for the cars are the same but the num- 
ber of cars in the area is limited to 30, 60, and 90, corresponding to a sparse, 
medium, and crowded traffic condition. In all cases, the MALS strategy is found 
to learn how to reduce the average queue length and improve the traffic speed 
in the network. For example. Figure 7 shows the average performances of the 
random control strategy and MALS strategy respectively over 10 runs in the 
crowded case, where the solid line represents MALS strategy and the dotted 
line represents random control strategy. It can be seen that the MALS strategy 
consistently learns and improves the traffic speed over 10,000 iterations. 



5 Conclusion and Future Work 

We have extended the framework of the multi-agent learning systems in [10, 12] 
to a general case and developed a simple communication architecture for these 
systems. The service provided by the communication architecture allows each 
agent to connect to the user interface, the application and the other agents. 
An application example shows that the communication architecture is reliable 
and efficient. Although the communication architecture is implemented using 
TCP/IP, it can also be implemented using RMI or DCOM, via binding the 
objects in the remote registry and making calls to the remote objects. 
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Abstract 



The production of a roster for the duties of ambulance crew is subject to a variety of 
practical, managerial and social constraints. This document describes the first steps 
of an investigation into the characteristics of the search space for such a problem and 
reports on the success to date of an evolutionary algorithm approach in finding an ac- 
ceptable solution to the problem. The visualisation method described is suggested as 
a quicker way of testing the appropriateness of representations than having to per- 
form multiple time consuming experiments. 



Section 1 - Introduction 

Recently, the decision was made to re-establish the ambulance service sub-station 
attached to the Victoria Hospital in Kirkcaldy. The previous arrangement was to provide all 
ambulance services from a central base in Glenrothes. ‘Front Line’ personnel for the new 
service comprises a group of eight staff, four of whom are designated paramedics and have 
received specialised training for their duties, and four staff who are designated medical 
technicians, also after appropriate training. This group of staff are required to provide twen- 
ty four hour a day, seven day a week coverage. Each ambulance is crewed at all times by a 
team consisting of one paramedic and one technician, working a twelve hour shift. Clearly 
such a coverage scheme requires careful arrangement to ensure that sensible rosters are pro- 
duced for all of the individuals concerned. There are a number of constraints on when indi- 
viduals are available to crew a vehicle. 

Given the nature of the duties that crew members undertake it is not a practicable 
option to allow double shift working, for both the safety of users of the service and the crews 
themselves. This complement of staff implies that a crew member is required to perform an 
average of three and a half shifts per week. In practice this translates to either three or four 
shifts in one seven day period. Experience has shown that regularly working excess shifts 
in a week to be both physically and emotionally draining, and this is to be avoided as far as 
possible. 

In addition to the practical constraints described, management have decided against 
forming permanent pairings. This means that each possible combination of paramedic and 
technician must work together at some stage in the schedule. This is seen to bring with it 
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the benefits of promoting ‘skill pull-through’ and ensuring that the relationships between 
paramedics and technicians remain productive. In the interests of ‘fairness’ it is desirable 
that each of the sixteen (four paramedics and four technicians) possible teams are active an 
equal number of times. 

A further complication arises from the ambulance staff themselves. Many of them 
have families, and are desirous of spending ‘quality time’ with them. This not only pro- 
motes development of the family unit, but also allows the staff to unwind after some of the 
more harrowing incidents that they are required to attend. Possibly the most important op- 
portunity is to spend time with a family is during the weekend, when children are not at 
school. 

It is clearly not possible to avoid weekend work, but the staff have a stated prefer- 
ence to either work a complete weekend, or to have no shifts in a weekend at all. The inter- 
mediate situation where only one shift is worked in a weekend is referred to in house as a 
‘ruined weekend’ . There is thus a request that single shift weekends are avoided by any sys- 
tem used to generate ambulance staffing rosters. 

There is also the notion of perceived ‘fairness’ within a roster. This is best viewed 
as a desire to have all members of staff working an equal number of day /night shifts and the 
same number of weekend/weekday shifts. This prevents any accusations of bias when slight 
imbalances are noticed in rosters. 

Staff leave requires no special treatment, as staff are seconded from the central am- 
bulance base in Glenrothes to fill in for scheduled holidays. This may cause slight imbal- 
ances in the frequency with which individuals are partnered, but this is not regarded as 
important. 

Section 2 - Approach 1 

Attempts to produce a roster by hand proved that the problem was quite difficult, and 
a computerised solution was sought. At first glance the development of a software system 
to automatically generate useable ambulance rosters within the constraints described in sec- 
tion 1 looks as though a simple depth first recursive tree search would be a feasible ap- 
proach, as the choice of the ‘next shift’ is so highly constrained that considerable pruning 
of the search tree would be possible. Unfortunately the effect of some of the constraints can 
only be assessed when the tree has been grown to some depth. This has the effect of making 
exhaustive search an non-viable option even if very fast computational machinery were 
available. 

This leads us into the area where a search based on evolutionary techniques suggests 
itself as a possibility. Even from the outset, the constraints placed on the system suggest that 
finding a solution will present a considerable challenge, but it was felt that given the past 
successes in solving schedule based problems with genetic algorithms (GAs), including 
those described in Langdon [Langdonl995] Wren and Wren [Wrenl995] and Fang et. al. 
[Fang 1993], there was a reasonable chance of success. 
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The alphabet initially chosen to represent the genetic material in the candidate so- 
lutions has sixteen characters, one for each of the different technician paramedic pairings. 
This was partly to allow easy checking to see if the partnership constraints were being met. 
The selection of the length of the genetic material (chromosome length) is also an issue, as 
the length of the roster is not specified. Under such circumstances an application of messy 
GAs [Goldberg 1990] may prove particularly appropriate, but for an initial investigation a 
fixed length representation was chosen. 

As there are sixteen teams to accommodate the fixed length was set at sixteen weeks, 
giving a pattern that repeats three times a year. Each of these sixteen week cycles would 
ideally have each partnership active fourteen times (seven on day shift and seven on nights). 
A year could thus comprise three of these cycles, leaving a gap of about four weeks. This 
is convenient for the disruption in pattern required for the Christmas/New Year period when 
the working requirements are subject to change. The festive period is a busy time for the 
emergency services. 

Note that if working to these sixteen week rosters that the ‘wraparound’ effect from 
the end of one cycle to the beginning of the next must be taken into account. Failure to do 
so may result in doubleshift working between the end of a cycle, and the beginning of the 
next. 



The first incarnation of genetic search was largely to gain insight into the character- 
istics of the space, and used a fairly traditional approach, in that each locus on the chromo- 
some was allowed to take any of the allowable alleles, a low mutation rate, uniform 
crossover [Syswerdal989] and binary tournament selection [Brindlel981] (with 100% 
chance of the better of the two candidates progressing). 

Following the success reported by Miller et. al. [Millerl995], Jones and Bradbeer 
[Jonesl994] and Chisholm and Bradbeer [Chisholm 1997] using small breeding pool sizes, 
a relatively small breeding pool of thirty was initially adopted. A random sample of five 
thousand individuals generated the distribution shown as figure 1 . This suggests that a large 
portion of the population have poor fitness, and leaves us hoping that there are very long (if 
thin) tails to the distribution. This type of distribution, allied with the fact that there are 
many constraints on the search space reinforces the suspicion that this will be a difficult 
problem for this brand of GA. It is worth commenting that there was little code produced 
for this initial system that was not to be of use in subsequent implementations. 

The fitness function used for evaluation purposes is based on the accrual of penal- 
ties, with different weights being assigned to breaches of different constraints. Each time a 
constraint is violated the integer representing the fitness of the candidate solution is in- 
creased. This means that in the reporting of results a low fitness number indicates a a good 
solution. 
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Initially the following penalties were allocated 



Constraint breach 


Penalty level 


Day/night imbalance 


1 


48 hour-F per seven days 


1 


Double Shift 


1 


Ruined Weekends 


1 



Table 1: Initial Penalty Scheme 

This apportionment of penalties implies that all breaches of constraints are equal, 
which is clearly an oversimplification of the problem. In reality, a sixteen week roster that 
produced one or two breaches of the 48 hour rule would probably be acceptable, whereas a 
scheme with the same number of double shifts would not be acceptable. Those constraints 
that must not be breached are usually referred to as hard constraints, while those for 
which minor or limited breaches could be tolerated are referred to as soft constraints. 
Usually a hard constraint violation would attract a higher penalty value than a soft penalty. 

Michchalewicz [Michchalewiczl994] provides a discussion of some of the soft con- 
straints that can appear in timetabling problems, and the contrasting hard constraints. 

The apportionment of relative penalty levels is an area of interest in its own right, 
and can either be static, or dynamic as can be seen for example in Eiben et. al. [Eibenl998] 
where different approaches are compared. 

As this is an initial study however we are more interested in the gross performance 
of the system, and merely note that an ideal solution to this problem would produce a zero 
penalty result. 

For more detailed study, the exact weight given to each class of breach would need 
more attention, possibly using an approach such as that detailed by Paechter et. al. 
[Paechterl998] where a ‘front panel’ is attached to the system allowing dynamic alteration 
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of the weights. It is certainly possible that different penalty levels could change the trajec- 
tory of the search. 

Fitness Distribution (30000 samples) 




Figure 1: Distribution of fitness for random sample using ‘unconstrained’ representation. 

This sample indicates that the average randomly generated individual breaches 
about four hundred and forty constraint requirements, and the bulk of the population is in 
this vicinity. It is difficult to believe that an acceptable solution will breach more than about 
twenty constraints, so that leaves a considerable challenge for the search mechanism. 

In order to progress the search, some sort of move operator must be employed. It is 
useful to have an indication of the likely effectiveness of operators. Random mutation is one 
of the operators traditionally used, and the effect of this operator in this representation in 
this problem is summarised in figure 2. Mutation in this case involves the selection of a sin- 
gle locus on the chromosome, and replacing it with a single randomly generated legal value. 
The figure is generated from a sample of 30000 individuals, each of which receives one mu- 
tation. The resulting change in fitness due to the mutation is noted against the fitness of the 
original. Rather than keep records of individual fitness numbers, the outcomes are grouped 
into ranges, and averaged. 

The graph shows the phenomenon of above average fitness individuals generally be- 
ing adversely affected by random mutations, whereas below average fitness individuals are 
on average improved. The kinks at the extremities can be explained as due to small sample 
sizes at the tails of the distribution. 
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No steps were taken to prevent the mutation replicating the original value. As an al- 
phabet of sixteen was used, approximately one in sixteen mutations result in no change. 
This may be viewed as reducing the slope of the graph slightly. 

A graph with a gradients low as this can be taken as indicating that the representation 
induces severe epistatic effects, or that the problem space has no structure. 

A series of runs (with a range of parameter sets) using this approach gave sufficient- 
ly poor results to confirm that a better representation should be sought. 



Average Change due to Mutation 




Figure 2: Average change in fitness under mutation. (3 0000 samples) 



Section 3 - Approach 2 

Instrumentation of the runs of the first approach seemed to indicate that the major 
source of penalty points was due to difficulty in maintaining the even spread between the 
number of times each team was rostered. Clearly, the next step was to reconsider the repre- 
sentation. As a way of reducing the imbalance penalties a permutation representation was 
employed, with each team appearing a given number of times within a sixteen week roster. 
This change largely fitted into the framework of the previous system, with relatively small 
code changes. As before, a sample of randomly generated individuals was evaluated to as- 
certain if any benefit accrued. The results are noted in figure 3. 
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Figure 3: Distribution of fitness for random sample using single permutation chomosome 
compared to unconstrained chromosome (30000 samples). 

Comparing figure 3 with figure 1 we see that the distribution has moved far enough 
to provide encouragement to code the rest of the system to perform further test runs. 



Effect of mutation on permutation representation 




Fitness 



570 
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Figure 4: Average change in fitness to permutation representation under mutation. (3 0000 

samples) 

Once again performing the change under mutation test, we see that the gradient of 
the response is very low, again leading us to suspect that there is a large amount of interac- 
tion left in the representation. 

Using PMX [Goldbergl985] as the crossover method, a number of test runs were 
performed. Various parameter sets failed to produce even near acceptable results. Again the 
representation of the candidate solutions permitted far too many constraints to be violated. 
It is believed that this in turn led to a highly modal search space that caused difficulties for 
the search algorithm. 



Section 4 - Approach 3 

The representation described in the previous section had the effect of forcing all can- 
didate solutions to obey the constraint requiring all team pairings to be equally represented, 
but the ‘equitability’ request is left to the evolutionary mechanism. In order to reduce the 
number of undesirable combinations further, a third encoding was devised. This was made 
up from four separate components (chromosomes), each encoding a permutation of teams. 
The first two, each of length sixteen, determine which weekend the team will work a day 
shift and which weekend they work a night shift. If a team is scheduled to work both day 
and night shift then this will be detected and accrue the associated penalty. This approach 
guarantees that not only does each team work an equal number of weekend day and night 
shifts, but also that the requirement to avoid ruined weekends is avoided. This renders half 
of the fitness function redundant, thus speeding the evaluation portion of the system by 
about 25%. Similarly the third and fourth chromosomes, each of length eighty encode five 
weekday shift and five week night shift appearances for each team. 

Figure 5 compares the fitness distribution of this representation, which on the basis 
of this admittedly small sample seems only a little better. Figure 6 examines the effect of 
the mutation operation. Examination of the slope of this graph in comparison with the re- 
sults from the previous two representations reveals that it is more pronounced. This is taken 
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as indication that there is less epistasis evident in the representation. Even so the slope is 
still relatively shallow, indicating that the problem is still ‘hard’ . 




■ Ffermitation 



-X-Lhconstrained 
-0— dOronriBome 



Fitness 

Figure 5: Distribution of fitness for random sample using 4 chromosome arrangement, com- 
pared to single permutation chomosome and unconstrained chromosome (30000 samples). 



Bfect of niJtation on 4 Qiornosorre permutation representation 




Figure 6: Average change in fitness to 4 Chromosome permutation representation under 

mutation. (30000 samples) 

Preliminary runs with this representation gave a series of results with fitness in the 
order of ten constraint breaches. Closer examination of the resulting roster however re- 
vealed that about three of these were breaches of the ‘no double shift’ constraint, and were 
thus not acceptable. Rather than try to recode the representation further to totally avoid the 
occurrence of double shifts, it was decided to rebalance the weight of penalty accrued by 
the various breaches. The value for double shift violations was increased to 25, in the hope 
that this would drive the search into eliminating double shifts. This of course alters the fit- 
ness distribution values already produced, but as we were now nearing the ideal solutions 
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this was felt to be a small price to pay. It is also possible or even likely that these changes 
will change the search space slightly, with a resulting change in progress 



Section 5 - Results 

After making the change described to penalty weightings given in the previous sec- 
tion, the system was run with various parameters. One of the first runs, using a pool size of 
20, PMX, binary tournament selection and a 10% mutation rate produced a solution with 
only 5 minor breaches of the 48 hours per week rule. This was felt to be a practicable solu- 
tion to the problem, and is shown in figure 7. Despite a certain amount of experimentation 
with parameters, so far no other solutions have been generated that have such an acceptable 
fitness value. 
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Figure 7 : Screen dump of best result so far. 



Section 6 - Conclusions 

The system, developed against a practical need and relatively undeveloped as it is 
has produced a workable roster for a highly constrained problem. 
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As there are a number of different local optima being found, it is possible to give the 
‘client’ a number of different candidate solutions. This will allow consideration of other 
‘social’ factors not explicitly mentioned in the initial outline of the problem. 

This approach of looking at the behaviour of the search space has saved a consider- 
able amount of time in running experiments and analysing the results from them. While the 
relative shortcomings of the first two representations are to an extent predictable, given the 
nature of the constraints, it is useful to be able to confirm this without expending large 
amounts of compute cycles. 

Section 7 - Further Work 

This document describes the first steps towards solving this problem. The authors 
are aware that many variants are possible, and even desirable. As mentioned in the body of 
the text, the length of the cycle is not clear, so an approach allowing a variable length cycle 
is one possible direction for further study. 

It would also be of interest to discover more about the modality of the search space, 
and it is planned to use a reverse hill-climbing technique, such as described by Jones 
[Jones 1995]. 

Producing more information on the sensitivity of the problem to different parame- 
ters, such as pool size and crossover mechanism remains an item on the agenda. 

It has been noted that the addition of heuristics could be used to improve perform- 
ance, possibly in a similar fashion to that reported by Hart et. al. [Hartl998]. 
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Abstract. Although there has been a wealth of work reported in the 
literature on the application of genetic algorithms (GAs) to jobshop 
scheduling problems, much of it contains some gross over-generalisations, 
i.e that the observed performance of a GA on a small set of problems 
can be extrapolated to whole classes of other problems. In this work we 
present part of an ongoing investigation that aims to explore in depth 
the performance of one GA across a whole range of classes of jobshop 
scheduling problems, in order to try and characterise the strengths and 
weaknesses of the GA approach. To do this, we have designed a configur- 
able problem generator which can generate problems of tunable difficulty, 
with a number of different features. We conclude that the GA tested is 
relatively robust over wide range of problems, in that it finds a reasonable 
solution to most of the problems most of the time, and is capable of find- 
ing the optimum solutions when run 3 or 4 times. This is promising for 
many real world scheduling applications, in which a reasonable solution 
that can be quickly produced is all that is required. The investigation 
also throws up some interesting trends in problem difficulty, worthy of 
further investigation. 



1 Introduction 

Since the first applications of GAs to scheduling problems, [4], there have been 
many reported applications of GAs to scheduling problems in general, with the 
jobshop problem receiving a great deal of attention, for example [2,8,9], and 
more recently [3, 13]. A large variety of representations and operators have been 
reported, each showing impressive performance of a GA on some small subset of 
benchmark problems. Often this performance is then extrapolated to claim that 
the GA in question is a good algorithm for solving job-shop scheduling problems. 
However, in order to properly evaluate the quality of a method, it is important 
to show that it works over a wide range of problems. This requires testing the 
method over an extremely large number of problems - the problems should be 
chosen such that they exhibit a variety of features, and should vary in difficulty in 
some configurable way. Rather than use benchmarks problems, it is more useful 
to use a parameterised problem generator which can generate problem instances 
at random, and in some tunable manner, so that many different instances of 
problem classes can be generated. 
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Therefore, in this article we describe a problem generator which we have used 
to generate instances of jobshop scheduling problems falling into 280 different 
problem classes. We present the preliminary results from an ongoing investiga- 
tion, which allows us to come to some general conclusions about the robustness 
of a GA as a technique for solving scheduling problems. We evaluate our results 
not only in terms of the capability of the GA to find the optimum solution to 
the problems we generate (which all have a known optimum value), but also 
in terms of its accuracy at producing ’reasonable’ quality solutions on a reli- 
able basis. For real-world problems, it is seldom necessary to produce optimum 
solutions to problems — “good enough fast enough” will generally suffice. There- 
fore, it is important to take this into account when making statements about 
the performance of the GA. 

We first describe the features of the problem generator, then present a sum- 
mary of the results from 28000 experiments on a range of scheduling problems. 
Some general conclusions about the performance of the GA are then drawn. 

2 The Problem Generator 

We designed a parameterised, tunable problem generator which generates a 
solution to a jobshop scheduling problem, i.e. a gantt chart, characterised by 
P{0,M,D,I,S), where O is the number of operations to be scheduled, M is 
the number of machines, D defines the distribution that the operation sizes are 
drawn from, (either Gaussian or uniform), I is the total amount of idle time per 
machine, and S is the amount of slack in the arrival and due dates of each job. 
The parameters are described in more detail below: 

Operations/ Johs/Machines The operations, O, are divided equally between the 
number of machines, to. A schedule is generated at random, by placing operations 
into the gantt chart. The operations are then assigned to jobs, j, such that each 
job is processed only once on each machine, and an operation of a job on a 
machine cannot begin until its operation on the previous machine has finished. 
Therefore, although the minimum number of resulting jobs is 0/M, the actual 
number depends on the exact manner in which the tasks are allocated to jobs. 

Sizes of Operations The size of each operation in O is drawn from either a 
uniform distribution between min and max, U (min, max), or a Gaussian dis- 
tribution of specified mean and deviation, G(m,d). 

Idle Time The amount of idle time on each machine is specified as a percentage 
of the total processing time of operations on that machine. The idle time is 
randomly and uniformally distributed between the actual operations. 

Slaek The maximum amount of slack in the arrival (A) and due-dates D of each 
job is also specified as a percentage of the total processing time (pt) of the job. 
If there is no slack, then the arrival date of the job is set to be equal to the 
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time slot at which the job is first processed in the generated schedule, and the 
due-date is set to the time-slot in which the job finishes. If slack is specified, 
then the arrival and due dates of each job are altered randomly such that 



A —>■ A — (random(0, S) * pt) 

D ^ D + (random{0, S) * pt) 

Therefore, no job is ever tardy, and the maximum tardiness objective, T^ax 
for each generated problem has an optimum value of 0. For each solution, there 
is also a known upper bound on the makespan of each problem — if the problem 
has no idle time, then the optimum makespan is known exactly. 

3 Experimental Parameters 

3.1 The Genetic Algorithm 

The GA used is HGA which was described by the authors in [6]. This GA 
outperformed other heuristic combination methods, and compared well to the 
most recently published results on a number of benchmark problems. It uses an 
indirect representation, in which each gene on the chromosome encodes a pair 
{Method, Heuristic). The method gene denotes the methodology that should 
be used to calculate a conflicting set of schedulable operations at each iteration 
of the algorithm. An operation is then chosen using the heuristic denoted by the 
heuristic gene. The method applied is either the Gifffer and Thompson algorithm, 
[5], which produces an active schedule, or a modified G&T algorithm which 
produces a non-delay schedule. 

The GA used a population size of 100, uniform crossover and a swap mutation 
operator. Each experiment is run for a maximum of 500 generations, or until it 
finds the optimum solution. The maximum tardiness of the best solution is noted, 
with the number of generations required to find it if the optimum was discovered. 
The best solutions for each of the 100 trials in each problem class are averaged, 
and we also record the minimum and maximum solution quality in each class. 

3.2 The Problem Classes 

In the experiments presented in this article we fix the number of operations, 
and hence the chromosome length, at 60. This facilitates a thorough and fair 
investigation of the effects of the other four parameters on problem difficulty. 
Furthermore, as each GA experiment uses a chromosome population of identical 
size and length, we can fix the “GA” parameters such as the number of genera- 
tions to be the same in all experiments. 

The other four parameters are varied as shown in table 1, resulting in a 
total of 280 problem classes, each defined by a tuple P{0,M,D,I,S). (For all 
experiments in which 5 > 0, then I is fixed at 0, and vice versa). For each of the 
280 problem classes, 10 problems are generated from different random number 
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seeds. We then run a GA 10 times on each problem instance, and using Tmax 
as the objective function (which we know to have an optimum solution of 0), 
average the results over the 100 experiments in that class. 

In the remainder of this article, the term problem class refers to the general 
class of problems defined by a tuple P{0, M,D,I,S). The term problem instance 
refers to one of the 10 problems generated for a problem class. 



Parameter 


Value 


Total Number of Operations 


60 


Number of Machines 


2,3,4,5,6 


Task Size Distribution 


U(0,10), U(0,30), U(0,50), U(0,100) 




G(50,l), 0(50,10), 0(50,20), G(50,40) 


Idle Time 


5%, 10%, 25% 


Slack 


10%, 25%, 50% 



Table 1. Experimental Parameters 



4 Results 

As space limitations do not allow us to present the results of all 280 experiments 
here, we attempt to present summaries of the findings and general trends ob- 
served, and discuss some individual problem classes in more detail. The complete 
set of results can be found at [1] . 

4.1 Overall Performance 

Firstly, we note that in every problem class, the GA is able to find the optimum 
solution for at least 1 instance of the problems in that class. On the whole, better 
performance is observed in the 140 experiments in which the operation sizes were 
distributed uniformally, (regardless of distribution size). For example, in 81% of 
problem classes that had a uniform distribution of task size, then running the GA 
on problem instances of that class resulted in the optimum solution being found 
in at least half of all experiments. This compares to a figure of 59% for problem 
classes with a Gaussian distribution of task size. These statements should be 
treated with some caution however as they may simply be an artifact of the 
parameters chosen to define the uniform and Gaussian distributions, and the 
two series of experiments cannot be directly compared. 

Turning our attention to those problem classes which are solved to “reas- 
onable” accuracy, we note that 55% of the problem classes with Gaussian task 
size distribution are solved to within 10 time units of the optimum solution. This 
increases to 96% for problem classes with uniformally distributed operation sizes. 

Tables 2 and 3 show the problem classes in which all generated instances 
where solved with 100% accuracy. There is no obvious pattern, except that it 
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Idle Time 


Slack 


Std.Dev 


Machines 


10% 


0 


10 


2 


25% 


0 


1 


3 


25% 


0 


1 


2 


25% 


0 


10 


2 


25% 


0 


20 


2 


25% 


0 


40 


2 


25% 


0 


40 


3 


0 


50% 


40 


6 



Table 2. Problems with Gaussian 
Operation Size Distribution which 
are solved with 100% accuracy 



Idle Time 


Slack 


Max Size 


Machines 


25% 


0 


10 


2 


25% 


0 


30 


2 


25% 


0 


50 


2 


25% 


0 


100 


2 


25% 


0 


30 


4 


0 


25% 


10 


5 


0 


25% 


10 


6 


0 


50% 


30 


6 


0 


50% 


50 


6 



Table 3. Problems with Uniform 
Operation Size Distribution which 
are solved with 100% accuracy 



appears that for perfect performance, either a large value of idle time or large 
slack is required in all cases, and that for Gaussian distributions of operation 
size, a large value of standard deviation helps. 

For problems with a Gaussian distribution of operation size, the worst per- 
formance is observed in 2 problem classes which both result in the optimum 
solution only being in 1 run of 1 of the 10 problem instances. These problems 
classes are P(60, 5, G(50, 10), 0, 0) and P(60, 6, G(50, 10), 0, 0). In the uniform 
case, the worst performance, again with only 1 optimum solutions is for a prob- 
lem class P(60,6,G(50),0,0). 



4.2 Number of Machines 

Intuitively, it would be expected that as the number of machines increases for 
a fixed number of operations, O, then the problems would become easier to 
solve. This is because the number of schedulable operations in the conflict set 
produced as a result of applying the G&T or non-delay algorithm must reduce 
as the number of machines increases. Table 4 shows the percentage of problem 
classes with m € 2,3,4, 5,6 in which the optimum solution was found. For a 
uniform distribution of jobs, the results are as expected. However, for the Gaus- 
sian distributions, we see exactly the opposite — i.e the percentage of optimum 
solutions decreases as the number of machines is increased. 





iNumber of Machines 




1 Slack Parameter 




2 


3 


4 


5 


6 




0.0 


0.1 


0.25 


0.5 


Gaussian 




56 


55 


55 


58 


Gaussian 


33 


32 


60 


82 


Uniform 


62 


65 


74 


75 


73 


Uniform 


46 


50 


76 


80 



Table 4. % of problem classes res- 
ulting in an optimum solution vs 
no. machines 



Table 5. % of problem classes res- 
ulting in an optimum solution vs 
slack 
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4.3 Slack 

Table 5 shows the percentage of problem classes with slack € (0.0,0.1.0.25.0.5) 
in which the optimum solution was found. For problems with Gaussian distribu- 
tions of task size, we observe that increasing the slack in due and arrival dates 
to 10% does not increase performance, as would be expected, and in fact results 
in a slight decrease. Increasing the slack to larger values however increases per- 
formance. This trend is not observed with uniform distributions, which tend to 
become easier to solve as the slack parameter is increased. 

Examining the effect of the slack parameter in more detail in the Gaussian 
experiments, we notice a correlation between the number of machines parameter 
TO and the slack S in determining solution quality. Table 6 shows the minimum 
value of the slack parameter S that was required for each set of problem classes 
with machine to before an improvement was observed in solution quality com- 
pared to the equivalent experiment with S = 0.0. Where no improvement was 
observed, adding slack had a detrimental effect on solution quality. 



Number of Machines 
Machines 


Minimum Percentage 
Slack Required 


Standard Deviation 


2 


0.5 


20,40 


3 


0.25 


all values of std. dev. 


4 


0.1 


all values of std. ded. 


5 


0.05 


all values of std. dev. 


6 


0.05 


all values of std. dev. 



Table 6. Minimum Value of Slack Parameter Required to Improve Solution Quality 



In problem classes where operation size was uniformally distributed, then 
adding slack generally increases performance. The only exceptions where a sig- 
nificant difference occurs are for the problem classes P{60, 2, [7(0, max), 0.0, 0.1), 
where max € (10, 30, 50, 100). 

4.4 Idle Time 

For problems with both uniform and Gaussian task distributions, adding idle 
time generally improves performance, as is seen in table 7, which shows the 
percentage of problem classes in which the optimum solution was found for 
various values of I. This is as expected — inserting idle time into the schedule 
allows some flexibility in the exact placement of operations in the schedule, 
without necessarily decreasing schedule quality. 

5 A Phase Transition in Problem Classes with Gaussian 
Distributions 

In the majority of problems with a Gaussian distribution of operation sizes, we 
notice an interesting transition in the difficulty of the problems as the standard 
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Idle Time 




0.0 


0.05 


0.1 


0.25 


Gaussian 


33 


48 


55 


85 


Uniform 


46 


65 


75 


89 



Table 7. % of problem classes in which the optimum solution was achieved vs idle 
time 



deviation of the distribution is varied. The GA performs best on those problems 
with large standard deviations. For very small standard deviations, i.e when the 
all tasks have very similar sizes, then the GA also performs reasonably well. 
However, for a range of values of standard deviation in the middle, perform- 
ance decreases considerably. For example, figure 1 shows an expanded graph for 
the problem class P(60, 5, G(50, sd), 0.0, 0.0), in which some extra experimental 
points have been added. A clear peak in difficulty is seen, centered around a 
standard deviation of 7. 




Fig. 1. Solution Quality vs Standard Deviation of Task Size Distribution 



This appearance of this phenomenum shows remarkably similar properties to 
earlier work performed by the authors in the timetabling domain. Work described 
in [11] showed that there was a clear phase transition in the performance of a 
GA on a sequence of (solvable) timetabling problems designed to be of increasing 
difficulty; the GA tested could solve very lightly constrained, and also very highly 
constrained problems, however, for moderately constrained problems, the GA 
would often fail to find a solution. The appearance of similar phase-transition 
regions has also been reported in other classes of constraint satisfaction problems, 
for example see [7, 10, 12]. 

In the timetabling case, it was noted that other (non-evolutionary) algorithms 
also failed on the same subset of problems, suggesting that it was not the GAs 
“fault” , but that the problems were intrinsically difficult. We have not yet invest- 
igated the performance of other non-evolutionary methods on these problems, 
but expect to see a similar pattern in performance. 
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1. Calculate the set C of all operations that can be 
scheduled next 

2. Calculate the completion time of all operations 
in C, and let m* equal the machine on which the 
minimum completion time t is achieved. 

3. Let G denote the conflict set of operations on 
machine m* - this is the set of operations in C which 
take place on m* , and whose start time is less than t. 

4. Select an operation from G to schedule 

5. Delete the chosen operation from G and return 
to step 1. 



Fig. 2. Giffler and Thompson Algorithm 



A possible reason, still to be looked at in more detail, is the use of the 
Giffler and Thompson (G&T) algorithm (or the modified non-delay version) in 
constructing the conflict sets of operations at each iteration. The G&T algorithm 
is shown in figure 2. For problems in which there is a large deviation in operation 
sizes, then it is possible that the conflict set is generally smaller at each iteration, 
and therefore it is more straightforward to choose the ’correct’ operation. When 
all the operations are of similar size, then it seems likely that the size of the 
conflict set is non-trivial, and hence it is more difficult to choose the ’correct’ 
operation. Figure 3 shows the size of the conflict set at each iteration, averaged 
over 100 runs of experiments in which the standard deviation of operations 
sizes was set to 5, and then to 40, using a constant mean of 50. Early in the 
scheduling process, there is a small region, highlighted on the figure, in which 
there a fewer items in the conflict set for the case when sd = 40. As we know that 
the placement of operations early in the schedule is crucial to the success of the 
algorithm, this may provide a clue, however the matter needs further attention. 



6 Problem Classes with Uniform Distributions 



As noted earlier, better results appear to be obtained when the operation sizes 
are uniformally distributed, compared to those problems with gaussian distri- 
bution of operation sizes. Solution quality tends to decrease as the range of the 
distribution increases. This information is summarised in table 8 which reports 
the percentage of optimum solutions obtained for instances of problems tested 
with each different range value. 
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Fig. 3. Size of Conflict Set For Different Distributions of Operation Sizes 





Maximum 




Task Size 




10 


CO 

o 


50 


100 


% optimum solutions 


80 


76 


71 


68 



Table 8. Percentage of problem instances resulting in an optimum solution vs max- 
imum task size 



7 Conclusion 

This article has presented some initial observations made whilst attempting to 
perform a systematic investigation on the performance of a genetic algorithm on 
a range of job-shop scheduling problems. The investigation involved a total of 
280 different problem classes, each containing 10 randomly generated problems. 

Initial findings suggest that although the GA does not find the optimum 
solution for all problem classes, for most instances of all problem classes it is 
capable of finding the optimum in at least 1 in 10 trials, and that the quality of 
solution is generally satisfactory. This suggests the GA appears to be a relatively 
robust method of tackling such problems, and that since runs are reasonably 
fast, running the GA several times to find a solution is a viable strategy. Some 
interesting trends have been observed — we now intend to try and understand 
and explain these trends, and to determine whether they are a feature of the 
problems themselves or due to the genetic algorithm itself. 

This study concentrated on problem classes which all contained a fixed num- 
ber of operations. Therefore, in order to complete the study, we will also use 
the generator to test GA performance on a sequence of much larger problems. 
Although we expect performance to degrade as the problems get bigger, it is 
fruitful to determine at what point this happens. 

Finally, further work is planned to compare the performance of other non- 
evolutionary algorithms on the same set of problem classes, to see if similar 
trends are observed, and to compare overall performance. These other methods 
will include constraint satisfaction techniques, and simple scheduling rules. 
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Abstract. Ant Colony Optimization is an evolutionary method that 
has recently been applied to scheduling problems. We propose an AGO 
algorithm for the Single Machine Total Weighted Tardiness Problem. 
Compared to an existing AGO algorithm for the unweighted Total Tar- 
diness Problem our algorithm has several improvements. The main nov- 
elty is that in our algorithm the ants are guided on their way to good 
solutions by sums of pheromone values. This allows the ants to take into 
account pheromone values that have already been used for making earlier 
decisions. 



1 Introduction 

Ant Colony Optimization (AGO) is an evolutionary metaheuristic to solve com- 
binatorial optimization problems by using principles of communicative behaviour 
found in real ant colonies (for an introduction and overview see [5]). Recently 
the AGO approach has been applied to scheduling problems, like Job-Shop [2, 7], 
Flow-Shop [13], and the Single Machine Total Tardiness problem [1]. Bullnheimer 
et al. [1] have compared an AGO algorithm with several other heuristics to solve 
the Single Machine Total Tardiness problem (e.g. decomposition heuristics, in- 
terchange heuristics and simulated annealing). They have shown that the AGO 
algorithm found the optimal solution of 125 benchmark problems more often 
than the other heuristics (these benchmark problems where generated with the 
same method from [12] as the benchmarks problems used in this paper ). 

In this paper we propose alternative and improved ways to solve the Single 
Machine Total Tardiness problem by AGO. Moreover, we also study the weighted 
version of the total tardiness problem. 

In AGO algorithms several generations of artihcial ants search for good solu- 
tions. Every ant of a generation builds up a solution step by step going through 
several probabilistic decisions until a solution is found. In general, ants that 
found a good solution mark their paths through the decision space by putting 
some amount of pheromone on the edges of the path. The following ants of the 
next generation are attracted by the pheromone so that they will search in the 
solution space near good solutions. In addition to the pheromone values the 
ants will usually be guided by some problem specihc heuristic for evaluating the 
possible decisions. 
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The approach used in [1] and [13] to solve scheduling problems with AGO 
algorithms is to use a pheromone matrix T = {Tij } where pheromone is added to 
an element Tij of the pheromone matrix when a good solution was found where 
job j is the ith job on the machine. The following ants of the next generation 
then directly use the value of Tij to estimate the desirability of placing job j as 
the ith job on the machine when computing a new solution. 

Here we propose a different approach. Instead of using only the value of Tij 
the ants use to compute the probability of placing job j as the ith 

on the machine. A problem with using only Tij can occur when the ant does 
not chose job j as the ith job in the schedule. Because, if the . . . 

values are small then job j might be scheduled much later than at the ith place 
(and possibly long after its due date). It is likely that this will not happen 
when using ■ Note, that this approach differs from nearly all other ant 

algorithms proposed so far, in that we base one possible decision of an ant on 
several pheromone values. The only other work that uses several pheromone 
values to estimate the quality of one possible decision is [11]. Moreover, we let 
the ants make optimal decisions when this is possible and use a heuristic that is 
a modihcation of the heuristic used in [1]. 

This paper is organized as follows. The Single Machine Total Weighted Tardi- 
ness Problem is dehned in Section 2. In Section 3 we describe an AGO algorithm 
for the unweighted problem. The pheromone summation rule is introduced in 
Section 4. Section 5 contains further variants and improvements. The choice of 
the parameter values of our algorithms used in the test runs and the test in- 
stances and are described in Section 6. The results are reported in Section 7. A 
conclusion is given in Section 8. 

2 The Single Machine Total Weighted Tardiness Problem 

The Single Machine Total Weighted Tardiness Problem (SMTWTP) is to Rnd 
for n jobs, where job i, 1 < i < n has a processing time pj , a due date dj, 
and a weight Wj , a non-preemptive one machine schedule that minimizes T = 
'YTj=i kkj • max{0, Cj — dj} where Cj is the completion time of job j. T is called 
the total weighted tardiness of the schedule. The unweighted case, i.e. Wj = 1 
for all J e {1, . . . , nj, is the Single Machine Total Tardiness Problem (SMTTP). 

It is known that SMTTP is NP-hard in the weak sense [8] and SMTWTP 
is NP-hard in the strong sense [10]. A pseudopolynomial time algorithm for 
SMTWTP in case that the weights agree with the processing times (i.e. pj < 
Ph implies Wj > w^) was given in [10]. Observe, that the last result implies 
that SMTTP is pseudopolynomial time solvable. For an overview over different 
heuristics for SMTWTP see [4]. 

3 AGO Algorithm for SMTTP 

The AGO algorithm of Bullnheimer et al. [1] is described in this section. The 
general idea was to adapt an AGO algorithm called AGS-TSP for the traveling 
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salesperson problem of Dorigo et al. [6] for the SMTTP. In every generation 
each of m ants constructs one solution. An ant selects the jobs in the order in 
which they will appear in the schedule. For the selection of a job the ant uses 
heuristic information as well as pheromone information. The heuristic informa- 
tion, denoted by rjij, and the pheromone information, denoted by Tij, are an 
indicator of how good it seems to have job j at place i of the schedule. The 
heuristic value is generated by some problem dependent heuristic whereas the 
pheromone information stems from former ants that have found good solutions. 
With probability qo, where 0 < go < 1 is a parameter of the algorithm, the ant 
chooses a job j from the set S of jobs that have not been scheduled so far which 
maximizes 



[njT hjf 

where a and /3 are constants that determine the relative influence of the 
pheromone values and the heuristic values on the decision of the ant. With 
probability 1 — go the next job is chosen according to the probability distribution 
over S determined by 



hkf 

The heuristic values rjij are computed according the Modihed Due Date rule 
(MDD), i.e.. 



max{T + pj , dj} 

where T is the total processing time of all jobs already scheduled. 

After an ant has selected the next job j, a local pheromone update is per- 
formed at element (i,j) of the pheromone matrix according to 



Tij = (1 - p) • Tij + p - To 

for some constant p, 0 < p < 1 and where 

1 

"^0 - 7T, 

m ■ Iedd 

and Tedd is the total tardiness of the schedule that is obtained when the 
jobs are ordered according to the Earliest Due Date heuristic (FDD), i.e., with 
falling values of l/dj. The value tq is also used to initialize the elements of the 
pheromone matrix. 

After all m ants have constructed a solution the best of these solutions is 
further improved with a 2-opt strategy. The 2-opt strategy considers swaps be- 
tween all pairs of jobs in the sequence. Then it is checked whether the so derived 
schedule is the new best solution found so far. 

The best solution found so far is then used to update the pheromone matrix. 
But before that some of the old pheromone is evaporated according to 
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Tij — (1 p) ■ Tij 

The reason for this is that old pheromone should not have a too strong 
influence on the future. Then, for every job j in the schedule of the best solu- 
tion found so far some amount of pheromone is added to element (ij) of the 
pheromone matrix where i is the place of job j in the schedule. The amount 
of pheromone added is p/T* where T* is the total tardiness of the best found 
schedule, i.e.. 



1 

Tij - Tij + P ■ — 

The algorithm stops when some stopping criterion is met, e.g. a certain num- 
ber of generations has been done or the best found solution has not changed for 
several generations. 



4 The Pheromone Summation Rule 

In this section we describe a new approach of using the pheromone values which 
is used in our AGO algorithm for SMTTP. In general, a high pheromone value Tij 
means that it is advantageous to put job j at place i in the schedule. Assume now 
that by chance an ant chooses to put some job h at place i of the schedule that 
has a low pheromone value Tih (instead of a job j that has a high pheromone 
value Tij). Then in order to have a high chance to still end up with a good 
solution it will likely be necessary for the ant to place job j not too late in the 
schedule when j has a small due date. To some extend the heuristic values rjij 
for I > i will then force the ant to choose j soon. But a problem occurs when 
the values Tij are small (because no good solutions have been found before that 
have job j at some place I > i). Then the product {rjij)“ ■ (Tij)^ is small and it is 
likely that the ant will not choose j soon. In this case the ant will end up with 
a useless solution having a high total tardiness value. 

To handle this problem we propose to let a pheromone value Tij also influence 
later decisions when choosing a job for some place I > i. A simple way to guaranty 
this influence is to use the sum of all pheromone values for every job from the 
Rrst row of the matrix up to row i when deciding about the job for place i. When 
using this pheromone summation rule we have the following modified decision 
formulas. An ant chooses as next job for place i in the schedule with probability 
qo the job j G S' that maximizes 



( 2 ) 

k = l 

and with probability I — qo job j G S is chosen according to the probability 
distribution over S determined by 
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(EUiMr-k-/ .3. 

5 Further Variations and Improvements 

In this section we describe further variations and improvements that we used in 
our AGO algorithm. 



5.1 Modified Heuristic 

A problem when using the heuristic values according to formula (1) is that the 
values of max{T + pj,dj} become much larger — due to T — when deciding 
about jobs to place at the end of the schedule than they are when placing jobs 
at the start of the schedule. As a consequence the heuristic differences between 
the jobs are, in general, small at the end of the schedule. To avoid this effect we 
used the following modified rj values 

max{T + pj,dj} — T ^ ^ 

For the weighted problem SMTWTP we multiplied every value on the right 
side of equation (4) with the weight Wj of job j. Note that jobs with a small 
weighted processing time pj /wj have a high heuristic value when T + pj > dj . 



5.2 Deterministic Scheduling Between Due Dates 

Consider the construction of a schedule for the unweighted problem SMTTP. 
Assume that some jobs have already been scheduled. Assume further that the 
sum T of the processing times of all jobs scheduled so far lies between some due 
date dj and a due date dh > dj and every other due date is smaller than dj or 
larger than dh- For this case it is easy to show that it is optimal to schedule all 
jobs with a due date < dj before scheduling a job with a due date > dh as long 
as the sum of the processing times of the scheduled jobs is at most dh - Moreover 
when there are several jobs with due date < dj it is optimal to schedule these 
jobs ordered by increasing processing times. If the ants apply this deterministic 
rule whenever possible we say that the ants work locally deterministic. Then the 
ants will switch between probabilistic and deterministic behaviour. 



6 Test Instances and Parameters 

We tested the different variants of AGO algorithms on 125 benchmark instances 
for SMTWTP of size 100 jobs that are included in the OR-Library [14]. These 
benchmark instances were generated as follows: for each job j G [1 : 125] an 
integer processing time pj is taken randomly from the interval [1 : 100], an 
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integer weight Wj is taken randomly from the interval [1 : 10] and an integer due 
date dj is taken randomly from the interval 



125 



Y^P^.(1-TF 

i=i 



ROD 

2 



125 



i=i 



ROD 
2 ^ 



The value RDD (relative range of due dates) determines the length of the in- 
terval from which the due dates were taken. TF (tardiness factor) determines the 
relative position of the centre of this interval between 0 and Xlj=i Pj ■ '1'^® values 
for TF and RDD are chosen from the set {0.2, 0.4, 0.6, 0.8, 1.0}. The benchmark 
set contains Rve instances for each combination of TF and RDD values. For the 
unweighted problem SMTTP we used the same benchmark instances but ignored 
the different weights. 

Our results for SMTWTP were compared to the best known results for the 
benchmark instances that are from [3] and can be found in [14]. 

The parameters used for the test runs are: a = 1, /3 = 1, p = 0.1, qo G {0, 0.9). 
The number of ants in every generation was m = 20. Every test was performed 
with 4 runs on every instance. Every run was stopped after 500 generations. 

We used a 2-opt strategy to improve the best solution that was found in every 
generation which differs slightly from the 2-opt strategy used in [1]. For every 
pair of jobs it was checked exactly once whether a swap of these jobs improves 
the schedule. A swap that improves the schedule was fixed immediately. Thus 
we tried exactly 4950 swaps per generation. 

In the following ACS-SMTTP (or short ACS) denotes the algorithm of [1] 
as described in Section 3 but with the new 2-opt strategy described in the last 
paragraph. Our algorithm ACS-SMTWTP-A is similar to ACS but uses the 
pheromone summation rule as described in Section 4. Algorithm ACS-SMTWTP- 
H is similar to ACS but uses the new heuristic from Section 5.1. Algorithm ACS- 
SMTWTP-D is similar to ACS but additionally uses the deterministic strategy 
from Section 5.2 for scheduling between due dates. Algorithms that use combi- 
nations of new features are denoted by ACS-SMTWTP-XYZ where X,Y,ZG {Y, 
H, Dj (e.g. ACS-SMTWTP-HY uses the new heuristic and the pheromone sum- 
mation rule). For shortness we write ACS-XYZ for ACS-SMTWTP-XYZ. 



7 Experimental Results 

The influence of the pheromone summation rule (called Y-rule in the following) 
and the modified heuristic was tested on weighted and unweighted problem in- 
stances. Since the parameter qo has some influence on the results we performed 
tests with qo = 0 and qo = 0.9. 

Table 1 shows the results for SMTWTP. The average total tardiness values 
found by the ACO algorithms for SMTWTP were compared to the average to- 
tal tardiness of the best known solutions that are from [3]. The average total 
tardiness per instance of the best solutions from [3] is 217851.34. Table 1 shows 
that ACS-YH performed better than ACS-H and also that ACS-Y performed 
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better than ACS (this holds for both cases qq = 0 and qo = 0.9). In all cases the 
difference of the total tardiness values compared to the best known solutions are 
at least 61.1% lower for the AGO algorithm with A-rule (79.5 for ACS- AH com- 
pared to 204.5 for ACS-H with qo = 0.9). Moreover, the AGO algorithms with 
A-rule found for more instances a better total tardiness than their counterparts 
without A-rule (at least 5.3 times as often). The differences of the total tardi- 
ness values compared to the best known values over the Rrst 200 generations are 
shown in Figure 1. The best solution of ACS- AH was found after an average of 
80 generations, which was after less than 3.5 seconds on a 450 MHz Pentium-II 
processor. 

Table 1 also shows that the AGO algorithms with modified heuristic per- 
formed in all cases better than their counterparts using the heuristic from [1]. 
For qo = 0.9 the advantage of the modified heuristic is smaller than for qo = 0 
(e.g. for qo = 0.9 ACS- AH has a 60.2% smaller difference to optimal total tar- 
diness than ACS-A compared to a 92.9% smaller difference for qo = 0). 



Table 1. Influence of pheromone summation rule and new heuristic on solution quality 
for SMTWTP. Total Tardiness: average difference to total tardiness of best found solu- 
tions from [3] (average over 500 test runs, f25 instances and 4 runs for each instance); 
Better: comparisons between ACS- AH and ACS-H (respectively ACS-A and ACS), 
number of instances with smaller average total tardiness (average over 425 instances 
and 4 runs for each instance). 



weighted 


ACS- AH 


ACS-H 


ACS-A 


ACS 


Total 

Tardiness 


go = 0 


f9f.8 


3024.7 


946.4 


9944.7 


go = 0.9 


79.5 


204.5 


200.0 


4498.6 


Better 


go = 0 


97 


2 


406 


0 


go = 0.9 


86 


46 


97 


3 



Table 2 shows the results for the unweighted problem SMTTP. The results 
are compared with the average of the best total tardiness values we found for 
the unweighted instances, i.e. 54309.5. Similarly as for the weighted problem in 
all cases the AGO algorithms with A-rule are better than their counterparts 
without A-rule. Also the modified heuristic performed better in all cases than 
the heuristic from [1]. 

Since the 2-opt strategy significantly influences of the quality of the solutions 
we also compared the ACS- AH with ACS-H when using no 2-opt strategy. The 
results can be found in Table 3 for SMTWTP and in Table 4 for SMTTP. The 
only case where ACS- AH performed not significantly better than ACS-H is the 
unweighted case with qo = 0.9. In this case ACS-H found a slightly better average 
total tardiness ACS- AH (difference is 331.5 for ACS-H and 332.3 for ACS- AH). 
On the other hand ACS- AH found for more instances better solutions than 
ACS-H (For 65 instances ACS- AH found better solutions than ACS-H whereas 
ACS-H performed better than ACS- AH for 33 instances). 
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Fig. 1. SMTWTP: Average difference to total tardiness of best fonnd solntions from 
[3] over the first 200 generations. 




Table 2. Inflnence of pheromone snmmation rnle and new henristic on solntion qnality 
for SMTTP. Total Tardiness: average difference to total tardiness of best fonnd soln- 
tions (average over 500 test rnns, 125 instances and 4 rnns for each instance); “Better” 
as in Table 1. 
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Table 3. Inflnence of pheromone snmmation rnle and new henristic on solntion qnality 
for SMTWTP when nsing no 2-opt. “Total Tardiness” as in Table 1; “Better” as in 
Table 1 bnt comparison between ACS-Afl and ACS-fl. 
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Table 4. Influence of pheromone summation rule and new heuristic on solution quality 
for SMTTP when using no 2-opt. “Total Tardiness” as in Table 2; “Better” as in Table 
1 but comparison between ACS-IfH and ACS-H. 
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The influence of the deterministic strategy for scheduling between due dates 
for SMTTP has only a minor influence on the results for the unweighted bench- 
mark instances from the OR-Library. The reason is that these instances have 
small gaps between the due dates. Thereby, the deterministic strategy does 
come into play only rarely. Hence, we created new test instances which have 
two neighboured due dates that have a large gap in between. We changed each 
of the problem instances from the OR-Library as follows. The jobs were ordered 
by their due dates and the due dates of jobs 41 to 59 were set to the same due 
date that job 40 has. The average of the best total tardiness values we found for 
these modihed instances was 56416.3. Table 5 shows for qo = 0, that ACS-27HD 
performed much better than ACS-27H and also that ACS-HD performed much 
better than ACS-H. For qo = 0.9 the ACS-AHD algorithm could not proRt from 
the deterministic scheduling between due dates. 



Table 5. Influence of deterministic strategy between due dates on solution quality for 
SMTTP and problem instances with modified due dates. “Total Tardiness as in Table 
2; “Better” as in Table 1 but comparison between ACS- AH and ACS-AHD. 
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8 Conclusion 

We have introduced a new method to use the pheromone values in an Ant Colony 
Optimization (ACO) algorithm for the Single Machine Total Weighted Tardiness 
problem. An ACO algorithm using this pheromone summation rule gives better 
solutions for 125 benchmark than its counterpart that does not use the summa- 
tion rule. This holds also for the unweighted total tardiness problem. Moreover, 
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we proposed a new heuristic that can be used by the ants when searching for a 
solution. For the unweighted problem we have shown that the AGO algorithm 
can proRt from ants that switch between a deterministic behaviour (in case that 
optimal decisions can be made) and the “standard” probabilistic behaviour. 
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Abstract. This paper describes a genetic algorithm approach for sequencing 
problems especially for job shop scheduling. In actual problems, setup time 
should be optimized. In order to reduce setup time, we developed a new genetic 
representation and an efficient crossover operator called Common Cluster 
Crossover (CCX). In our representation, chromosomes represent the shift of the 
order in a sequence. To preserve sub-sequences in crossover operations, we im- 
plemented the process to identify the cluster of the sub-sequences and applied 
CCX that exchanges common clusters between two parents. The approach was 
tested on two standard benchmarks and applied to an audio parts manufacturer. 
CCX achieved remarkable results on the actual job shop scheduling problem. 



1 Introduction 

Job Shop Scheduling Problems (JSSPs) are solved by assigning jobs to resources. 
Those assignments are called operations. Jobs are basic tasks. Through a chain of 
operations, materials are processed into intermediate parts, and the parts are proc- 
essed into products. Consequently each job completes. The sequence among opera- 
tions of each job must be restricted. At any time, one resource can process only one 
operation. 

JSSPs are known as the most difficult problems and they cannot be solved in a 
practically timely manner because they are NP-hard. The genetic algorithm (GA) 
which is recognized as the effective tool to solve combinatorial problems has been 
applied to JSSPs in various approaches [1][2][3][4]. 

S. Cagnoni et al. (Eds.): EvoWorkshops 2000, LNCS 1803, pp. 297-306, 2000. 

© Springer-Verlag Berlin Heidelberg 2000 




298 T. Masam et al. 



In this paper, we propose a new genetic sequencing approach and a new crossover 
operator that works efficiently to minimize setup time on JSSPs. When GAs are used 
in sequencing approach, several specially designed crossover operators such as the 
partially mapped crossover (PMX)[5], the order based crossover [6], and Cl [7] are 
used to eliminate infeasible sequences. Those operators preserve some of sub- 
sequences but frequently destroy others. For example, the order based and Cl pre- 
serve the complete sub-sequences inherited from one parent but just a relative 
position from the other. 

In most practical JSSPs, sub-sequences are very important because setup time is 
required for resources to change setups from the one for a certain item to the one for a 
new item. Successive production of the same items reduces the setup time. Thus, we 
consider that preserving good sub-sequences that reduces the setup time is important 
to reach the good solution quickly. 

In order to allow offspring to inherit sub-sequences from the both parents, we 
propose a new genetic sequencing representation and the Common Cluster Crossover 
(CCX). In our framework, chromosomes represent the shift of the order in a sequence. 
We added the process to identify the cluster of the sub-sequences, and crossover 
operation is applied to the common clusters between two parents. 

In Section 2, we describe the details of the representation and CCX. In Section 3, 
we report the results of two computational experiments, one is for FT 10x10 and 
FT20x5 and the other is for an audio parts manufacturer's scheduling problem. Then, 
we conclude with the results in Section 4. 

2 Preserving Sub-sequences 

2.1 Genetic Representation 

In our approach, chromosomes represent the shifting distance of the order in a 
sequence. One chromosome corresponds to one resource, and one individual consists 
of a group of chromosomes. The number of chromosomes in one individual equals to 
the number of resources in a schedule. 

Each chromosome is decoded to the sequence of the jobs on the resource. Let us 
consider the case in which y jobs are processed by a certain resource. The length of the 
chromosome corresponding to the resource is j-\. The z-th allele of the chromosome 




0, 1,2, ...,j-3,j-2,j-l 
0, 1,2, ...,j-3,j-2 
0, 1,2, ...,j-3 




Chromosome 



gene 1 


gene 2 


gene 3 




gene i 




genej-1 



Fig. 1. The alleles. 
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can take an integer [Oj'-i-l] as shown in Fig. 1. 

The decoding process of the chromosome to the sequence is shown in Fig. 2. In the 
figure, the chromosome is {5, 4, 1,0,1}. At first, the sequence is initialized as 
{1,2,3,4,5,6}. The value of the first gene is five, and this means that the first symbol 
of the sequence is to be exchanged for the symbol on five steps right. Thus, symbol 
‘1’ and ‘6’ are exchanged and the sequence becomes {6,2,3,4,5,1}. The value of the 
second gene is four, and this means that the second symbol of the sequence is to be 
exchanged for the symbol on four steps right. Thus, symbol ‘2’ and ‘1 ’ are exchanged 
and the sequence becomes {6,1,3,4,5,2}. Repetition of this procedure for the whole 
genes leads to the sequence {6,1,4,3,2,5}. 



Chromosome 



0. A chromosome to be decoded. 




Sequence 



1. Initialize the sequence 




Chromosome 

Sequence 

Chromosome 

Sequence 

Chromosome 

Sequence 

Chromosome 

Sequence 




Sequence 



3. The sequence decoded from the chromosome. 




Fig. 2. Decoding process of the chromosome to the sequence. 
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2.2 Common Cluster Crossover (CCX) 

The random selection of the crossover point is typically used in the conventional GA 
approaches. Even if the new genetic representation we mentioned above is adopted, 
the random selection can destroy the sub-sequence. In order to select the effective 
crossover point, we propose the Common Cluster Crossover (CCX). CCX identifies 
the clusters of sub-sequences and exchanges the common clusters between two 
parents. 

A cluster consists of the genes linked together. Fig. 3 shows the example of clusters. 
As mentioned in Section 2.1, each value of a gene is the shifting distance. The arrows 
in the figure indicate the link of the shifts. In this case, the chromosome has two 
clusters. As shown in the figure, a cluster can consist of several separate segments. 

CCX exchanges the common clusters of two parents. Fig. 4 describes CCX process. 
In this example. Parent 1 has two clusters, A and B. Parent 2 has four clusters, C, D, E, 
and F. Two dotted arrows indicate the common boundaries of clusters. CCX 
exchanges the common clusters that are separated by the common boundaries and 
create two offspring. 

Fig. 5 shows the example of CCX. One offspring is reproduced ftom Parent 1(P1) 
and Parent 2 (P2) by CCX. PI has two clusters, A and B. P2 has four clusters C, D, E, 
and F. Since two boundaries match between PI and P2, CCX exchanges clusters at 




Fig. 3. The cluster of sub-sequences. 




Fig. 4 Common Cluster Crossover (CCX). 
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Fig. 5 Common Cluster Crossover. 

these points. 

I would like to emphasize that the offspring inherits the sub-sequences from both 
PI and P2. In this case, the sub-sequence (2, 9, *, *, *, *, *, *, 10, 1,11) was inherited 
from PI and ( *, *, 8, 6, 4, 5, 3, 7, *, *, *) from P2. We believe that this feature of 
CCX is very effective to optimize and reduce the setup time in JSSPs. 

There are two cases in which CCX cannot be applied. The first case occurs when 
no position of the boundaries matches between two parents. The second case occurs 
when both parents consist of one cluster. In other words, there is no boundary. In the 
first case, CCX is applied according to the clusters of one parent and those of the 
other parent are ignored. We call this crossover quasi-CCX. In the second case, 
conventional one-point crossover is applied. Naturally, preservation of sub-sequences 
is not guaranteed in these cases. 

3 Computational Results 

3.1 Standard Job Shop Scheduling Problem 

The GA based on our representation and CCX has been implemented in Java, 
JDKl.1.8 on the Pentium 550MHz PC with Windows 98. 
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The GA was tested on two standard job shop problems, Fisher and Thompson’s 
FT 10x10 and FT20x5 [8], as a benchmark. These two problems have been widely 
used by many researchers as benchmarks. The objective of the benchmark problems is 
to minimize makespan. The optimal solutions are 930 on FTlOxlO and 1165 on 
FT20x5. In the field of GAs, on FT 10x10, Yamada[9] first found the optimum 930 
four times among 600 trials in 1992. Kobayashi[10] founds 930 fifty-one times 
among 100 trials in 1995. On FT20x5, Mattfeld[ll] and Lin[12] found optimum 1165 
in 1994, 1997 respectively. 

It is known that the optimal schedules of makespan are among active schedules. 
The method to generate active schedules (GT method) was proposed by Giffler and 
Thompson [13]. Some other approaches use the GT method [4][10][14]. We also used 
the GT method to interpret the job sequence into an active schedule. 

We adopted the ranking selection [15] and the elitist policy [16]. Mutation changes 
the value of a gene into a random integer following uniform distribution [0, y-z-l] 
where 7 is the number of jobs and i is the loci of the gene. 

In our experiments, the population size was set to 200 and termination was set to 
250 generations. Crossover rate was set to 1.0 and mutation rate was set to 0.01. It 
took about 400 seconds to run one trial on FTl 0x10 and 450 seconds on FT20x5. 

Fig. 6 and Fig. 7 show the frequency distribution of solutions (makespan) in 100 
trials on FTlOxlO and FT20x5 respectively. The approach we proposed could not 
obtain optima but produced highly efficient solutions. I must point out that in 
practical use it is necessary to create good enough schedules reliably and fast rather 
than find optimal solutions. 

As mentioned in Section 2.2, our crossover operation consists of three kinds of 
crossover. CCX is applied when there are common clusters between two parents. 
Quasi-CCX is applied when no boundary matches between two parents. One-point 
crossover is applied when both parents consist of one cluster. Fig. 8 shows a temporal 
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Fig. 6 Frequency distribution of solutions on FTlOxlO. 
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Fig. 7 Frequency distribution of solutions on FT20x5. 




1 50 100 150 200 

Generation 

Fig. 8 A typical temporal change of the number of three kinds of crossover. 



change of the numbers of three kinds of crossover in a certain trial on FTlOxlO. In the 
first generation, about 30% of crossover operations was CCX, 55% was quasi-CCX, 
and 15% was one-point crossover. The proportion of CCX increased gradually, and in 
the end of iteration, CCX increased to 80%, quasi-CCX decreased to 15%, and one- 
point crossover decreased to 5%. 

We surmise that the proportion of CCX increased since good sub-sequences as 
building blocks were created, maintained and combined during iteration. 



3.2 Application to an Audio Parts Manufacturer 

The GA approach we proposed was applied to the scheduling of an audio parts manu- 
facturer. Since the scheduling problem involves setup time reduction, we can test the 
optimization ability of CCX by measuring the reduction in setup time. 
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The manufacturer is a subcontractor of an electrical appliance manufacturer and 
receives production orders once a month. According to the production orders, it 
makes its production schedule once a month. 

It has fifteen work centers as resources. One hundred workers are involved in the 
production. Its main products are audio speakers and headphones, ft produces about 
7500 products per month. Lot sizes are calculated by a MRP system. Its monthly 
production schedule typically consists of 300 operations. In the production, setup time 
is required for resources to convert from the production of a certain item to another 
item. 

Formerly, the manufacturer had used a dispatching-based scheduler. In order to 
reduce setup time, the worker had to modify the schedule created by the scheduler, ft 
took about three to four hours to modify with a Gantt chart editor. To refine the 
scheduling process, our GA-based scheduling system was introduced. The objective 
of the manufacture is to minimize total delivery delay time, the number of delaying 
jobs, and total setup time. 

First, we used total setup time solely as the fitting function in order to test the abil- 
ity of CCX. The test data consist of 84 jobs and 300 operations. For comparison, we 
also applied order based operator [6] as the traditional operator to the same problem. 
Population size was set to 50 and termination was set to 40. To create schedules in a 
reasonably short time. We set the values smaller than on the benchmarks, ft took 
about 80 seconds to run one trial with both CCX and order based. Fig. 9 shows the 
frequency distribution of solutions (setup time) in 100 trials. Obviously CCX is supe- 
rior to the traditional crossover operator in minimizing setup time. 

Next, we tested weighted sum of total delivery delay time, the number of delaying 
jobs, and total setup time as the fitting function on the same data. Population size and 
termination were also set to 50 and 40 respectively. Table 1 shows the best and the 
average of CCX and order-based in 100 trials. The result of a legacy dispatching- 
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Fig. 9 Frequency distribution of solutions on setup time minimizing. 
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Table 1. Application to an audio parts manufacturer 





Total setup 
time (minutes) 


Total delivery 
delay time(days) 


The number of 
delaying jobs 


CCX 


best 


5165 


7.16 


16 


average 


5766 


8.40 


18 


Order- 

based 


best 


5575 


7.21 


15 


average 


6085 


8.55 


18 


Legacy scheduler 


8425 


8.87 


25 



based scheduler is also shown for comparison. CCX approach successfully optimized 
the whole indices, especially setup time. What needs to be emphasized here is that in 
order to minimize total setup time, sub-sequences are quite important because 
successive production of the same kind of items reduces the total setup time 
remarkably. 

4 Conclusions 

In this paper, we proposed a new genetic representation and Common Cluster 
Crossover (CCX). Chromosomes represent the shifting distance of the order in a 
sequence and are decoded to the sequence of the jobs. CCX exchanges the common 
clusters of two parents. Integrated use of our representation and CCX allows offspring 
to preserve sub-sequences. Our discussion shows that this feature is highly effective 
in solving JSSPs especially problems to optimize setup time. 

We tested our approach on the standard JSSPs FTlOxlO and FT20x5 and found the 
approach was effective. 

Since we developed CCX in order to optimize setup time in particular, we also 
applied our new approach to an audio parts manufacturer. Setup time should be dealt 
with on the problem. First, only setup time was used as the fitting function to test the 
ability of CCX, and CCX successfully minimized setup time. Next, the fitting 
function was set to minimize the weighted sum of the total delivery delay time, the 
number of delaying jobs, and the total setup time. The proposed approach 
successfully optimized the JSSP based on the actual data. 
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Abstract. This paper examines two techniques for setting the parameters of an 
evolutionary Algorithm (EA). The example EA used for test purposes 
undertakes a simple scheduling problem. An initial version of the EA was 
tested utilising a set of parameters that were decided by basic experimentation. 
Two subsequent versions were compared with the initial version, the first of 
these adjusted the parameters at run time, the second used a set of parameters 
decided on by running a meta-EA. The authors have been able to conclude that 
the usage of a meta-EA allows an efficient set of parameters to be derived for 
the problem EA. 



1. A Description of the Problem 

The use of Evolutionary Algorithms (EAs) for solving timetabling and scheduling 
problems has become commonplace in recent years [1], [9]. This paper examines two 
methods for optimising the parameters used by the algorithm. Although the concepts 
and methods used for developing and optimising the evolutionary algorithm are 
intended to be general, for the purposes of presentation within this paper they will be 
applied to a simple scheduling problem. The scheduling problem under consideration 
requires a number of jobs to be processed through a factory. Each job has start and 
end times that form its time window. Within this window the job must “visit” all the 
resources required for this job, within a specified order. Each resource is mutually 
exclusive (ie only one job may make use of a resource at any one time). 

The various conflicting constraints for the scheduling task can be divided into two 
categories hard constraints and soft constraints, as shown in table 1. A hard constraint 
is one that must be satisfied in order to produce a feasible schedule, soft constraints 
alter the quality of the schedule. 



2. A Description of the EA 

The general concepts of evolutionary algorithms are well understood. Each individual 
requires a representation (a genotype) that may be decoded into a specific schedule (a 
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phenotype). Each gene within the genotype represents the start time of the event 
(stored as a real number). Within each genotype the event genes make up a 
chromosome, the event genes are grouped so that all the genes for a particular 
resource are stored together in order of start time. This representation may be 
considered a direct representation, in that the genes within the chromosome are a 
direct representation of the finished schedule. The mutation and crossover operators 
act directly on the items in the schedule. With an indirect representation the genes 
would represent a set of instructions for building the schedule [10], thus further 
processing would be required to convert the genotype into a schedule. 



Table 1. Hard and soft constraints 



Constraint 


Type 


Event must start after the previous event on this resource has finished. 


Hard 


Event must start after the previous event in this job has finished. 


Hard 


The first event must start after the jobs “available” time 


Hard 


The last event must finish after the Jobs “due” time 


Soft' 


The event should take place in normal working hours 


Soft 


The event should take place on week days 


Soft 



In the course of these experiments 3 data sets were utilised, they may be 
summarised as: 

Data set 1: 35 Events, within 16 Jobs using 4 Resources, over a 9 day period 

Data set 2: 41 Events, within 15 Jobs using 8 Resources, over a 4 day period 

Data set 3: 50 Events, within 18 Jobs using 8 Resources, over a 4 day period 

Each resource has a ‘queue’ of events, each event has a start time set to the nearest 
minute. Thus for an event being placed within a schedule being built for a 4 day 
period there are 1440*4 possible values for the start time (there are 1440 minutes in a 
day). Thus if there were 10 events scheduled to use a resource over a 2 day period the 
number of potential queues for that resource would be ( 48 * 1440)*10 = 619200 
different queues. The total number of potential solutions to the scheduling problem is 
the product of the size of the potential queues for each resource. The vast majority of 
these schedules are infeasible, and the restricted mutation operator discourages their 
production. 



2.1 Basic Operations 

Within the system two-point crossover, based on the Davis system is used to create 
children, based on exchanging start time genes. The child replaces one of the 
randomly selected parents. The EA uses steady state population to reduce complexity. 



* A job which is not completed until after the end of its time window may not be 
desirable, but will still result in a schedule that is feasible. 
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Use is made of elitism, this ensures that the best individuals of each generation are 
preserved for inclusion in the next generation. 

The mutation operator considers each member of the population for possible 
mutation, the chance of a specific member being selected is based on the mutation 
rate which is expressed as a percentage. For instance with a mutation rate of 35% 
there is a 35% chance of any individual being selected. When an individual is 
selected, one gene is selected for mutation with a selection pressure of 1/L (where L is 
the length of the chromosome). 

The fitness function is based on the constraints outlined in section 1, with the 
satisfaction of hard constraints being given greater rewards than the satisfaction of 
soft constraints. The fitness for each individual event is calculated using the values 
shown in table 2. 



Table 2. Initial weight values used in the fitness function 



Constraint 


Reward 


Current event is “placed” (does not conflict with any other event) 


120 


All events that make up the current job placed 


100 


Current event takes place before the job is due for completion 


50 


Current event takes place after the job is “available” 


50 


Event takes place between Monday and Eriday 


25 


Event takes place before “night time” 


4 


Event takes place after “day time” 


4 



The EA was allowed to run until all the events were placed (ie all the hard 
constraints are satisfied). There may arise situations where there is no feasible 
schedule for a given data set (ie if 25 hours of work are scheduled to take place in one 
day) therefore a limit is set on the total number of generations that may be calculated. 
This limit is calculated as no of events *150. Through initial experimentation it was 
found that allowing the EA to continue mnning for a short period after a feasible 
solution had evolved allowed further improvement with regards to the soft 
constraints. The number of generations used for this run-on period was calculated as 
total number of events * 5. 



2.2 Improving the Quality of Schedules using Lamarckism 

To improve the quality of schedules constructed by the system, the authors 
implemented Lamarckian writeback [11],[5]. Lamarckian theory suggests that 
knowledge acquired by individuals during their lifetime may be passed genetically 
onto their descendants. This theory has since been discredited by biologists who 
maintain that crossover and random mutation are the main influences on evolutionary 
development. The type of improvement made to a schedule is illustrated in figure 1, it 
may be accepted that any solution where events within a job “overlap” is not feasible 
and thus of no practical value. The work of [10] outlines the use of Lamarckianism 
within an EA used for solving the Napier University timetabling problem. 
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2.3 Targeted Mutation 

In [10] use is made of what is known as “Targeted mutation”. This modifies the 
mutation operator to increase the chances of genes with specific characteristics being 
selected for mutation. The actual characteristics that are used to select a gene for an 
increased chance of mutation will vary depending on the application. In most cases 
the genes that will have the extra bias in favor of mutation will be those that exhibit 
some defect requiring improvement. An extra weight value will be added to the 
chances of genes exhibiting that defect. In the case of the problem under 
consideration the authors decided to implement the bias so that genes relating to 
events breaking a hard constraint have a higher chance of being selected for mutation. 
The mutation function was been modified so that if a gene selected for mutation 
represents an event that does not break a hard constraint there is a chance (expressed 
as a percentage) that another gene may be selected. Table 3 shows the results of using 
a 40%, 50%, and 60% bias against the mutation of placed genes. 




In this extract we can observe 5 events 
within 2 jobs. In two instances events 
begin before the previous event has 
ended. 




After writeback the two events using the 
third resource have had their start times 
modified to ensure that they begin after 
the previous event within their job 
sequence has ended. 



Fig. 1. An example of part of a schedule before write back(left) and after (right) 

writeback 

2.4 Premature Convergence and Regeneration 

Within the EA it is desirable to maintain a diverse population. When a good solution 
is found, this individual will begin to dominate and thus reduce the bio-diversity 
within the population. The convergence of many population members to a similar 
point within the search space is feature of all EAs. It is desirable though, to try to 
prevent convergence until the EA beings to converge on the optimum solution. One 
method of eliminating the premature convergence effect is to implement a 
regeneration operator. Within the regeneration operator a number of individuals have 
their genes randomly reset. Thus in the case of the scheduling algorithm under 
consideration a number of members of the population will have the times of their all 
events randomly mutated. Consideration must be given to the frequency of the usage 
of the operator and the percentage of the population involved. If the regeneration 
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operator is utilised too frequently then the population will not have time to evolve, if 
it affects too many individuals with a high fitness then the EA may loose useful 
genetic material. 

The operator as implemented initially, reinitialises the lower 50% of the population 
randomly every 50 generations. The regeneration operator in practice applies the 
mutation operator to all of the genes in the schedule, thus creating a new, random 
individual. The times taken to produce a feasible schedule using regeneration are 
shown in tableS. It may be seen therefore that using regeneration offers a substantial 
(approximately 40%) saving in time. 



3. Setting the EA Parameters 



The parameters used within the EA to specify population size, crossover rate, 
mutation rate and regeneration have the biggest bearing on the success or failure of an 
EA. Conventionally the EA designer usually sets these parameters to values decided 
with reference to previous experimentation. The authors decided to investigate two 
alternative methods of arriving at a set of parameters. A version of the EA using the 
initial parameters was constructed and will be referred to as version CP. 



Table 3. Time in seconds to produce a feasible schedule using data set 2 



Run 


1 


2 


3 


4 


5 


Average 


No Bias 


570 


118 


19 


502 


1457 


768.5 


50% Bias 


no 


780 


1617 


280 


1599 


563.3 


40% Bias 


238 


353 


534 


1614 


163 


716.3 


60% Bias 


51 


1619 


137 


934 


108 


646.9 


Regeneration 
and 50% bias 


57 


181 


472 


228 


1631 


343.6 



3.1 Modifying the Parameters at Run Time 

The first method investigated is based on a simplified version of the system described 
in [13] and [5]. Each of these parameters starts off set to a low value, when the 
individual at the head of the population (ie the most fit individual) changes the 
parameter that invokes the operator responsible for creating that individual is 
increased slightly. This is simplified from the original system proposed by Davis [13] 
which not only took into account the most fit individual, but also the operators used to 
create the ancestors of that individual for several generations previous to the current 
generation. 

A new version of the EA incorporating the ability to modify its parameters at run 
time was constructed. Whenever the most fit individual changes the parameter 
controlling the creating operation is modified to increase the usage of the operator that 
caused the improvement. For instance if a new most-fit individual is created by the 
mutation operator then the mutation rate is increased by 0.1%, in the event of the 
cross over operation creating a new most-fit individual then the cross-over rate is 
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increased by 0.1%. In the event of the regeneration operator being responsible then 
the frequency of the regeneration operator is increased by 1 generation. A new 
version of the EA implementing this operator was constructed, it will be referred to as 
VPl. Initial values for the mutation rate and crossover rate are 75% and 50% with the 
regeneration interval being set at 50 generations, as used in CP. The results obtained 
when utilising data set2 may be see in table 4. 

In order to attempt to decrease the time taken to build a schedule, it was decided to 
modify the starting values, thus allowing the EA itself to make one or the other 
become the dominant operator. A second version (VP2) with starting mutation and 
crossover rates were set to 10% and the starting regeneration interval was set to 1000 
generations, thus no operator is allowed to dominate at the start. 

Table 4. Fitness of schedules generated using data set 2. (CP - Constant parameters; VPl - 
Variable parameters, starts with 75% crossover and 50% mutation rates; VP2 - Variable 
paramters starts with 10% crossover and mutation rates) 



Run 


1 


2 


3 


4 


CP. 


34162.9 


34019.4 


34193.2 


34062.4 


VPl 


34598.1 


34487.4 


34497.6 


34352.3 


VP2 


34214.1 


34382.8 


34424.7 


34372.8 



From the figures in table 4 it may be seen that version VPl gives the highest 
average fitness when building schedules based on data set 2. The results would 
suggest that in order to be effective the run time parameters must have starting values 
that are proven to provide an efficient result when used without modification. It was 
found that when starting with low initial values the EA is unable to progress due to 
the lack or crossover, mutation or regeneration. 

Based on the results obtained from the testing of versions VPl and VP2, 
optimising the parameters at run-time does not appear to give any advantages over 
using parameters that the user has arrived at using “trial and error” methods. The 
reasons for failure may be connected with the initial parameter values. If the 
parameters are initially set to small values (as in version VP2), there is a lack of 
activity within the GA, and the parameters cannot increase, and thus increase the 
activity within the algorithm until new most-fit individuals are being created. 



3.2 Using a Meta-Level EA to Optimise the Parameters 

The parameters of an EA may themselves be optimised by using another EA [13], [4], 
known as a meta-EA, to attempt to evolve a set of efficient parameters. Within this 
meta-level EA the genes represent the parameters of the EA that we wish to optimise 
(hereinafter referred to as the problem EA). The fitness function of the meta-EA is 
based on the results of running the problem EA with the parameters stored in each 
individual within the meta-EA population. 

The parameters of the problem EA to be optimised are listed in table 5. The fitness 
of each individual within the meta-EA was calculated by running the scheduling-EA 
using the parameters encoded within that individual, the fitness was calculated as the 
fitness of the final schedule less a time value. 
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Table 5. Parameters used for the meta-EA 



Parameter 


Value 


Population size 


10 


Elite size 


1 


Mutation Rate 


40% 


Crossover Rate 


50% 



During initialisation the first 2 members of the population were set to equal the 
parameters used with version CP and the remaining members were initialised 
randomly. 

In order to ascertain the ability of the meta-EA to evolve an efficient set of 
parameters the following experiment was carried out over 6 days. Six identical 
workstations were allowed to run the meta-EA for approximately 136 hours, each of 
the EAs was mnning completely independently. In total 3961 different configurations 
were evaluated, this would mean each PC evaluated on average 660 different 
configurations, which would be 12.5 mins per evaluation. 



Table 6. Parameters requiring optimisation by the meta-EA 



Chromosome 


Min 


Max 


Note 


Pop size 


1 


200 




Elite size 


0 


10 


If 0 then elitism is switched off. 


Mutation rate 


0 


100 


% chance of any one individual being mutated. 


Crossover rate 


0 


100 


% chance of any individual being selected for 
crossover. 


Bias mutation in 
favour of fitness 


False 


True 


If true provide increase chances of being 
mutated to the most fit examples 


Regenerate time 


0 


100 


No of generations between re-generation 


Regenerate PC 


0 


100 


Percentage of population to re-generate. 



In all of the runs the meta EA fitness initially increased rapidly, before slowing. It 
should be remembered that the graph covers a period of approximately 6 days. If it 
had been practical to have allowed the meta-EA experiment to continue the graph 
suggests that some, if not all of the fitness rates would have continued to grow. 

After the meta EA experiment had finished the parameter set was extracted from 
the individual with the highest fitness. This is shown in table 7. The percentage of the 
population to be regenerated has been decreased and the interval between 
regeneration increased from the settings used in version CP of the GA. The 
population and the elite sizes have both been increased. These parameters have been 
incorporated in a new version of the EA known as MP. It is now possible to compare 
version MP with version VPl and version CP, the results are shown in table 7. 
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Table 7. :The parameters evolved using the meta-GA 



Parameter 


Value 


Crossover Rate 


68 


Mutation Rate 


30 


Regeneration Interval 


74 


Regeneration % of population 


29 


Population size 


95 


Elite size 


4 


Bias mutation in favour of fitness 


Yes 



Table 8. A comparison of the fitness value of schedules 



Run 


1 


2 


3 


4 


CP 


34500.08 


34715.8 


34162 


32891 


VPl 


34485.08 


34598.1 


34487.4 


34518 


MP 


34545.05 


34454.8 


34435.8 


34611 



4. Conclusions and Future Work 



4.1 Some General Conclusions on the EA 

The representation used is a basic direct representation. Although many EA 
scheduling techniques, make extensive use of memetic algorithms, the basic 
scheduling problem outlined in this report was solved without recourse to a purely 
memetic algorithm. A number of changes from the biological model were necessary 
to allow the system to produce schedules with a high fitness in a reasonable time. 
These changes were the use of a steady-state population, the use of restricted mutation 
and the use of Lamarkian writeback. 

The mutation operator was extensively modified and it was restricted to only 
mutating event start times to within the start and end time for that job, rather than 
anywhere on the schedule. The mutation operator was also given a bias to discourage 
the mutation of genes that represented already placed events. It may be concluded that 
the mutation operator works most effectively when it is restricted from producing 
schedules that are obviously infeasible. The mutation operator’s power stems from its 
ability to create changes that are not influenced by existing genetic material within the 
population. This ability to make “radical” changes can cause a bigger decrease than 
increase in fitness, if the changes are detrimental to the overall problem that the EA is 
attempting to solve. 

The directed mutation operator showed a significant improvement, the average 
times to build a feasible schedule decreased from 768.5 to 563.3 seconds for versions 
with no bias in favour of badly placed events, and 50% bias respectively. The 
standard deviation in time taken without bias was 632.3 seconds, but adding a 50% 
bias, this decreases to 584 seconds. To further examine the effects of targeted 
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mutation two more versions of the EA were constructed with 40% and 60% bias 
against the mutation of placed events. A more detailed comparison of typical runs is 
shown in figure 2. 

The fitness function is of paramount importance to the EA, and a subtle alteration 
to the fitness function will radically affect the output obtained from the EA. The exact 
values used as weights within the fitness function were established by “trial and 
error”. It was not possible to allow the meta-EA to optimise the fitness function 
values during the meta-EA experiment. If this had taken place it is likely that the 
meta-EA would have altered the fitness function values to give a high fitness 
regardless of the actual fitness of the schedule. It is acknowledged that a more 
efficient set of weights may exist. Some research into establishing an optimum set of 
weights for a fitness function within a drafts program [4] has been carried out with 
considerable success 

The use of the re-generation operator provided a marked improvement. Initially 
this operator appeared to be very disruptive and could possibly cause the loss of 
genetic material. The re-generation operator requires skill in its usage, if too many 
individuals are re-initialised or the operator invoked too frequently then useful genetic 
material will be destroyed. Eurther research to find if there is an optimum percentage 
of population to re -initialise and an optimum frequency of use would be desirable. 
The meta-EA lowered the percentage of the population to be re-initialised to 29%. 
The re-generation interval was increased to 74 generations. 



4.2 Some Conclusions on Parameter Optimisation 

Eigure 3 shows the fitness growth rates for typical mns using versions CP, VPl and 
MP, when mnning data set 2. Version CP represents the EA running with parameters 
decided by the user using “trial and error”, version VPl represents the best of the 
versions that allowed the parameters to be optimised at mn time, finally version MP 
represents the result of the meta-EA. 

When trying to pick a “best” version of the EA it is important to notice that 
different versions display different qualities. Eor instance version CP produced 
schedules with a slightly higher average fitness (for data set 3, see table 10) than MG, 
but the time required to produce the schedules is significantly less (see table 11). It is 
the conclusion of the authors that version MP appears to represent the best “trade-off’ 
in terms of achieving an acceptable level of feasible schedules, achieving a higher 
average fitness than the versions and completing the runs in significantly less time. 
Versions VPl and VP2 were unable to produce schedules that were efficient when 
compared with the results obtained using CP and MP (see table 9). The main reason 
for this appears to be due to the subtle nature of the links between the parameters. 
Adjusting the parameters based only on the most fit individual in the population is not 
necessarily productive. The entire population contributes to each other fitness through 
crossover therefore although one operator may appear to be dominant in terms of 
creating the individual with the highest fitness and this operator may not be dominant 
throughout the population. 
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Fig. 2. Fitness Growth with 0,40,50 and 60% mutation bias using data set 2 

A major conclusion that may be drawn from the work is that the optimisation and 
selection of the EA parameters may be best carried out by a meta-EA. In this case the 
meta-EA parameters recorded a small but significant increase in performance from 
those obtained using the more traditional “trial and error” methods. Within version 
CP, the version that used the “trial and error” parameters 86% of all the EA runs 
resulted in feasible schedules, using version MP that increased to 93%. 

It may be seen that using parameters derived by the meta-EA it is possible to 
achieve a faster and higher growth rate. Optimising with the meta-EA gives superior 
results to those achieved using parameters decided by the user (version CP) or by 
modifying the parameters at run time (VPl and VP2, see figure 4) 
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Fig. 3. Eitness growth comparison for typical runs of versions CP, VPl and MP 
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Table 9. Total jobs placed (averaged over 10 runs) 





Data set 1 (16 jobs) 


Data set 2 (15 Jobs) 


Data set 3 (18 jobs) 


CP 


16 


14.7 


17.7 


VPl 


15.8 


14.4 


17 


VP2 


15.9 


12.1 


13.6 


MP 


16 


15 


17.8 



Table 10. Schedule fitness (Averaged over 10 runs) 





Data set 1 


Data set 2 


Data set 3 


CP 


74626.5 


34433.383 


53698.60 


VPl 


74500.13 


34260.041 


53628.33 


VP2 


74592.81 


33950.86 


53009.92 


MP 


74600.81 


34478.634 


53681.26 



Table 11. Time taken for run (seconds averaged over 10 runs) 





Data set 1 


Data set 2 


Data set 3 


CP 


207.2 


805 


1388.3 


VPl 


214.5 


678.4 


1260.9 


VP2 


412.1 


1657.6 


2431 


MP 


154.3 


367.5 


338 



400 -- 
350 -- 
300 -- 
250 -- 
200 -- 




□ CP 
nvpi 

□ VP2 

□ VP3 
■ MP 



CP VP1 VP2 VP3 MP 



Fig. 4. Average time (seconds) to build a feasible schedule 



4.3 Future Work 

The authors plan to apply meta-EA principles to the optimising of an EA based 
system used to solve a routing problem. Recent work into differential evolution has 
raised the possibility of using differential evolution to optimise the weighting values 
used within the fitness function [2]. It may also be possible to use differential 
evolution to optimise the parameters for the whole EA. A major problem with the 
meta-EA is the processing time required, differential evolution may achieve similar 
results to the meta-EA, but with fewer CPU cycles being required. 
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Abstract. We evolve a robotic controller for a four-legged real robot enabling it 
to walk dynamically. Evolution is performed on-line by a linear machine code 
GP system. The robot has eight degrees of freedom and is built from standard 
R/C servos. Different walking strategies are shown by the robot during evolu- 
tion and the evolving system is robust against mechanical failures. 



1 Introduction 

Robots on legs constitute both one of the largest potentials and one of the largest chal- 
lenges for intelligent robotic control. Applications are numerous in all environments 
accessible to humans and animals but inaccessible to wheeled autonomous agents. In 
general the flexibility of the legged robot increases with decreasing number of legs, 
unfortunately so does also control complexity. It is less complicated to control a robot 
with six legs or more, since the robot can have four legs on the ground all the time 
providing stable static balance [2,4,9]. Already the fourlegged cas e becomes more 
difficult. A fourlegged creature can crawl like a turtle with partial support by its body 
or by a tail. In this way it is possible to walk on four legs with static balance. Static 
balance means that the agent is in balance at all moments scarifying some efficiency 
and flexibility by dragging their body over the ground. However, it is more advanta- 
geous in general to use a dynamic walk where the agent will fall if interrupted in the 
middle of a movement. 

We have evolved the first controller for a fourlegged robot, learn using a genetic 
programming system and a real robot. The evolution of behavior passes through sev- 
eral stages starting with simple paddling behavior, continuing through crawling, 
"camel walk", and finally galloping with dynamic elements in the walk. 



2 Experiments 

Since we are using a real robot the experiment consists of one hardware part, the 
robot, as well as a software part, the GP system. There is also a communication part of 
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the system, sending control command to the actuators and receiving feedback from the 
sensors. Most experiments were run on a long time scale of about 20 hours and for this 
reason we needed a special automatic system that monitors the experiment and pulled 
the robot back if it had advanced too far. 



2.1 The four legged robot 

The four-legged robot is built with eight standard R/C servos as actuators, see Figure 
1. The servos are of one of the smallest available but still have a momentum of 11 
Ncm. The servos are controlled by a standard servo controller card, which allows a PC 
with a serial port to control up to eight servos. No input sensors on the dog itself are 
needed for this experiment. 




Figure 1: The four-legged robot dog. 

Fitness is generated through a computer mouse, which the robot drags behind itself. 
Positive reinforcement is given by dragging the mouse in the forward direction. Drag- 
ging backwards gives a negative fitness signal. Just moving the robot gives a small 
positive contribution to fitness. The experiment set-up consists of a rectangular box 
(0.3mX1.2m) where the agent can learn forward walking. If it reaches the end of the 
box then it is automatically pulled backwards to the other end by an electric motor. 
Single experiments ran automatically with this set-up for more than 18 hours. 



2.2 Control system 



The control system is built around a linear genetic programming system evolving 
binary machine code [1,5]. The evolved structure is a linear string of instructions op- 
erating on a set of registers of a register machine. The control system is executed on a 
PC and normal mouse communication is used for feedback while the servos are con- 
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trolled through serial communication with a servo control card. The inputs to the 
evolved programs are placed into the registers of the register machine (processor) 
while the output vector is whatever remains in the registers upon termination of the 
program. Each individual that is evaluated is iterated eight times. In each iteration the 
eight outputs are sent as angles to the servos and the previous output vector is sent as 
input to the next iteration enabling a sequence to be evolved. Fitness is measured after 
the eighth iterations (and basically consists of the number of "ticks" that the mouse has 
moved forward). Some individuals are removed before evaluation, such as those, 
which do not move the servos at all. 




Figure 2: Feedback through computer mouse 




Figure 3: Cables to PC and power supply 



The population consisted of 100 individuals and the genetic operators where cross- 
over and mutation. Crossover was performed both as two-point string crossover and as 
homologous linear crossover [3,7]. The system used a small tournament for selection 
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and operates under steady state, see Figure 4. The function set consisted of arithmetic 
operators and square root. A single constant of 0.5 was used in the terminal set. 



Robot 



GP-system 



r 




Figure 4: On-line GP system 

Our method for using GP with a real-time application is based on a probabilistic sam- 
pling of the environment [6]. Different solution candidates (programs) are evaluated in 
different situations. This is unfair because a good individual dealing with a hard situa- 
tion can be rejected in favor of a bad individual dealing with a very easy situation. For 
instance, an individual that gets stuck near the wall of the box and does a good job of 
moving away from the wall but advances little in the forward direction might get a 
low score while a poor individual in the middle of the box might perform better. Our 
experience is, however, that a good overall individual tends to survive and reproduce 
in the long term. The somewhat paradoxical fact is that sparse training data sets or 
probabilistic sampling in evolutionary algorithms often both increase speed toward the 
goal and keep the diversity high enough to escape local optima during search. 



3 Results 

Evolving walking behavior, through coordination of eight servos and without any 
domain knowledge or guidance and where the only feedback is from a computer 
mouse on a rod, is not a trivial problem. The first emerging behavior is usually some 
kind of chaotic paddling, which slowly moves the robot forward even though it is 
barely visible. A common local optimum after the paddling is a strange strategy, 
where the robot stands upright on all legs carefully balancing and moving quickly 
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back and forth without really advancing the robot. It is unclear how this strategy 
emerges but it could be the result of some lack of symmetry in the hardware and 
measurement system. The next strategy is often some kind of crawling similar to that 
of turtle. Other observable behaviors are a "camel walk" where the legs move in paral- 
lel and pair wise on each side. The most efficient strategy for walking that has 
evolved is also the most difficult to learn and galloping only appears after many hours 
of training. Here both the front legs and back legs are parallel and the robot often 
stands up on its back legs before pushing forward. It uses the dynamic interaction with 
the heavy mouse lode to achieve maximum speed forward. 

Emergence of behavior does not always pass through all of these behaviors but usually 
most of them appear in order. 

Summary of common evolution of behavior over time could look like: 

1 . chaotic paddling behavior 

2. crawling 

3. "camel walk" 

4. galloping with dynamic elements in the walk 

Like many evolutionary control systems we observe some robustness against me- 
chanical failure. Figure 5 shows the evolution of speed over 111 generations and 15 
hours. During evolution a tree servos broke down but the control system adapted and 
managed to increase the speed after the failures, see Figure 5. Servo breakdowns oc- 
curred in generation 20, 75 and 100. The figure clearly indicates how fitness at first 
decreases rapidly but the system manages to relearn and adapt to the changed (dam- 
aged) hardware configuration. 




Figure 5: The velocity (in m/h) of the dog when controlled by the (currently) best 
individual during 117 generations. Servo failure occurred in generation 20, 75 and 
100 . 
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4 Future Work 

Our intention is to use the results obtained with the four-legged robot for a larger hu- 
manoid project, see Figure 6. Here the aim is bi-pedal walking, which is harder than 
four legged walking and demands much more emphasis on balance. Preliminary work 
on the ELVIS humanoid confirms the feasibility of the approach [8]. 




Figure 6: ELVIS humanoid a bipedal walking robot 

Making the robot fully autonomous is another ambition for the future. That would 
mean that the experiments will not be hindered by all the cables and total autonomy 
would also be interesting from a more philosophical standpoint. Eigure 7 shows a 
chaotic robot made of "garbage" which has learned to move by the same mechanism 
as the four-legged robot but which is fully autonomous. The on-board GP system runs 
on a small, embedded PIC-chip and evolves binary code for this tiny processor. Our 
plan is to move this system to the walking experiments. 
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Figure 7; An autonomous chaotic robot made from "trash" with an on-board GP sys- 
tem in an embedded chip. 



5 Summary and Conclusions 

We have evolved the first controller for a four-legged robot, which learn on-line using 
a genetic programming system and a real robot. The evolution is of behavior passes 
through several stages starting with simple paddling behavior, continuing through 
crawling, "camel walk", and finally galloping with dynamic elements in the walk. A 
high degree of robustness is seen for mechanical failure - the system is observed to 
adapt to changes in the mechanical configuration due to component failure. 
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Abstract. The GLEAM algorithm and its implementation are a new 
evolutionary method application in the field of robotics. The GLEAM software 
generates control code for real industrial robots. Therefore GLEAM allows a 
time related description of the robot movement (not only a static description of 
robot arm configurations). This internal representation of primitive move 
commands is mapped to a representation of move statements of an industrial 
robot language, which can be loaded at the robot control and executed. 



Introduction 

There are many different ideas and procedures about Genetic Algorithms and 
Evolutionary Programming, but only some of these lead to implementations which 
solve „real world problems" for industrial applications. GLEAM {Genetic hearning 
Algorithm and Methods) is an algorithm based on evolutionary strategy, which was 
used to implement a software tool for solving useful and not only academic problems, 
see [1] and [2]. 

The aim of the GLEAM application to industrial robots was an implementation 
with respect to practical requirements and restrictions to proof the GLEAM method. 
Therefore a first prototype implementation was done at ABB in Vasteras (Sweden) 
and a second one with improved facilities at Daimler Chrysler in Berlin (Germany). 
Both implementations include the output of robot language code for the specific 
industrial robot (ABB IRB 2400 and KUKA KR 6). 



The Evolutionary Algorithm GLEAM 

The principles of GLEAM including its genetic operators and genotype representation 
are formulated with close connection to the biological evolution (see detailed 
description in [3]). The genotype of GLEAM is called an „action“, which represents 
one step of a plan to be executed. The plan execution is done by simulation as part of 
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the evolution algorithm to get a value for the „fitness“ of the plan. In particular 
GLEAM generates a sequence of basic actions or statements, which are the elements 
of the plan, which is a member of the evolution population. A plan represents directly 
the genetic information, see [1]. The purpose of the plan is not of interest for the 
evolution itself, therefore the kernel of GLEAM including the evolution algorithm was 
applied to different problems with minor changes. For example, these actions can be 
the basic commands of a simple robot controller or allocation steps to reserve a 
machine in a production plan. 

The GLEAM method was implemented for several applications: 

• planning collision free moves for industrial robots, see [4] and [11] 

• generating production plans solving the job-shop-problem, see [5] 

• process scheduling in chemical industry, see [6] 

GLEAM is implemented as well in Pascal as in C on different hardware platforms, 
e.g. on PC, work station, and a parallel computer system. It is portable, because the 
implementation is structured modular and consists of the following main parts: 

Basic machine : E.g. initialization, data structure construction and management, 
error handling 

Optimization kernel : Evolution functions for mutation, recombination, 
population management 

Simulation and evaluation : Execution of plans, criterion’s check, restrictions, 
fitness calculation 

Overall control : Management of parameter input, display of results, 
interrupting simulation or optimization and others 




Implementation effort to apply 
GLEAM to another application 
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User interface : Criterion’s priority and fitness definition, target definition, 
simulation visualization, plan description and others 

Robot code generation module , generates a robot program part including the 
move statements 

Simulation and evaluation need the main amount of implementation effort to apply the 
GLEAM implementation to other applications. It means on the other side, that due to 
the generality of the method a new application requires less implementation costs 
because the basic machine, optimization kernel and overall control are not much 
effected. 



Application of GLEAM to Robotics 

The first application of GLEAM to robotics was performed because of two reasons: 

1 . The author is an expert in industrial robot programming and control (Convenor 
of an ISO working group), therefore a praxis oriented work was performed, see 

[7] 

2. The calculation for controlling an industrial robot to move on a predicted 
trajectory without collision is very complex but the result is easy to evaluate: 
everybody can see, if the movement is collision free and takes a shorter time. 

The application of GLEAM was something like a benchmark especially to show, that 
GLEAM is a powerful planning and optimization tool for the control of dynamic 
processes like the movement of industrial robots. 

The GLEAM method applied to robot collision free moves doesn't perform an 
explicit search of the configuration space, because the search parameters don't include 
configurations, like in [8] or [9]. With GLEAM the move statements for performing 
the collision free move are generated directly. The GLEAM method avoids the 
problem of calculating a smooth path between the configurations and of building up 
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and storing the configuration space, see [10]. The collision free path is optimized hy 
criterion’s selected hy the user, e.g. the criterion could be a short Cartesian path. Other 
criterion’s like energy or move execution time are integrated into the optimization 
process. The criterion’s could be contradictionary. Therefore every criterion has a 
priority and the results of the optimization reflect the different priorities. 

The evolution performed by GLEAM starts by generating action res. primitive 
statement sequences for controlling the robot movement. These basic actions for the 
robot moves look like follows: 

- move robot axis k with a velocity of r degrees per second with an acceleration of 

b degrees per seconfi (Evolution parameters are k, r, and b) 

- stop robot axis m with a slow down of n degrees per second^ (Evolution 

parameters are m and n) 

The fitness of a sequence can be calculated as a function of the following criterion’s: 

• position precision , i.e. the distance of the end point of the movement and the 
given target point 

• orientation precision , i.e. the difference between the planned and reached 
orientation of the gripper 

• orientation changes , i.e. the orientation moves of the gripper to reach the target 
orientation 

• quality of the move trajectory, i.e. the difference between the move path and a 
straight line 

• length of the move trajectory 

• fastness of move execution, i.e. the duration of movement 

• program length , i.e. the number of actions res. statements required to perform 
the robot move 

• Intermediate points , i.e. the robot move passes specified intermediate points 

• economy , i.e. the energy needed for the move execution 

Before the evaluation, the statement sequence is proofed for its plausibility, for 
example: a stop-axis-statement before a start-move-axis-statement is not meaningful 
(and therefore canceled). After the plausibility-check the evolution continuous by the 
mutation of the evolution parameters of the statements and by recombination. As the 
sequence of elementary moves is essential for the resulting overall movement there 
are some mutations altering only the sequence of actions. 

The complete sequence of actions controls the robot move to a point near the target 
position without collisions. It is assumed, that the robot control is based on a fixed 
control cycle time, e.g. of 50 ms. Every control statement is executed by the 
simulation tool with respect to this cycle time, which can be defined as a control 
characteristic. The simulation performs the robot move and stores the axis values for 
every control cycle. To measure the distance from the target position and the path 
length, the general forward transformation from robot to Cartesian coordinates is 
applied. 

The GLEAM method is able to develop a statement sequence for any kind of 
robot. A software tool ROBMODEE (Robot Model Definition) was implemented 
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providing a graphic supported robot definition. The user can build a robot with up to 
16 rotational robot axes, which can be directed into the x- or z-coordinate direction (in 
zero position), and which can rotate about the x-, y- or z-coordinate axis. 

The user can define obstacles in the robot movement area to show collision 
avoidance. The defined robot model, the obstacle definitions and the (predefined) 
action definition is called the "action model". It can easily changed or extended by the 
user and stored for further experiments. 



Genotype Representation and Data Structure 

The action of GLEAM is one gene and consists of the action code and a number of 
parameters. The parameters can be of integer, real or character type. The action code 




and parameter definition is stored in an (readable) file and can be change very easily. 
The number of actions, which build up a member of the evolution population, is not 
limited. The length of such an action chain is flexible and an optimization criteria of 
the evolution. 

GLEAM was designed for the optimization of dynamic processes with respect to 
the natural evolution. Therefore the evolution process operates about genes res. plans 
of variable length. Due to this attribute the task of recombination is more complicated 
than in evolution algorithms of other authors. The most important improvement of 
GLEAM is the introduction of the concept of so called „sections“. A section is formed 
by a (variable) number of genes res. actions as a substructure of the plan, i.e. the 
action chain is partitioned into segments. A segment can be regarded as a chromosome 
of the biological genetic information. So, one member, that means the action chain of 
the plan, consists of segments, and each segment of a number of actions. 

There are new defined evolution operators (for mutation and recombination) to be 
applied to the sections, like delete a segment or move a segment to another place in 
the action chain. This concept enables the recombination of „good“ sub-structures of a 
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problem solving process to speed up the evolution. If two plans are merged by the 
recombination, the combination of a segment of plan A with a segment of plan B can 
be treated as a combination of two sub-solutions. If the two sub-structures res. 
segments are good in single, they both together in the resulting plan will give a much 
higher fitness value than the fitness of the both parent plans. To come to such a good 
solution only by mutation of the genes takes probably much longer time. The concept 
of segments (the „chromosomes“) and actions (the „genes“) was very successful for 
planning and optimization tasks of dynamic processes like robot moves. 

Similar to the „traditional“ artificial intelligence language LISP, the data structure 
of GLEAM is based on dynamic lists. The action chain starts with a header node 
storing all necessary information about the action chain, e.g. chain length (e.i. number 
of actions and segments), fitness value, and others. The header is followed by actions, 
all linked by pointers. An action chain can grow or be reduced, it depends on the 
evolution. Of course, if an action chain (that means a problem solution) gets a good 
fitness value, and another chain gets the same fitness rank but consists of a lower 
number of actions, the last one is better and will survive. 
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Genotype Oriented Code and Robot Code Generation 



The genotype oriented code of GLEAM for robots consists of primitive move 
commands. The code is simulated during the evolution (without any output, only the 
fitness value is calculated during the simulation), and if the user wants to see the result 
of the planning and optimization process, the code of successful action plans is 
simulated with graphical display of the robot movements. 

As mentioned above, the robot control simulation calculates a move step by step, 
see [11]. Between every calculated via position of the trajectory a user defined cycle 
time has passed. The move description by primitive commands (used internally by 
GLEAM) has to be transferred to code formulated in a robot programming language, 
but programming the same robot move. A new module „robot code generator“ was 
implemented, which calculates the parameter for a move programming by robot move 
statements. 

After every control cycle the values for the robot axes and the Cartesian 
coordinates for the tool center point TCP are calculated and transferred to the robot 
code generator (if the user has marked „code generation“ with the help of menu input). 
The first step of the code generation is the declaration of position data and an 
assignment of values. This position data consists of the intermediate points of the 
move and the target point. The target point is given by the user in Cartesian 
coordinates, the code generator calculates the values for the parameters to describe the 
robot specific orientation and configuration data. After the data definition, a simple 
main function is generated consisting of move commands and referring to the position 
data generated before. The robot code can be downloaded to the robot control and 
executed. 

Example to program the ABB IRB 2400 robot: 

! COMMENT Position list 

! COMMENT with the move position for every cycle 

! COMMENT (representing the trajectory) : 

MODULE demol 

VAR robtarget target : = [[-1647.0,-604.2,1374.7], 

[0.12367,0.69621,0.12 367,-0.6 9621] , [-2, 0,0,0] , 

[9E9, 9E9, 9E9, 9E9, 9E9, 9E9] ] ; 



CONST jointtarget PI : = [[ 0.5, 45.3, 45.3, 

0.0, -45.3, 0.0], [9E9, .... , 9E9] ] ; 

CONST jointtarget P2 : = [[ 0.7, 45.3, 45.3, 

0.0, -45.3, 0.0], [9E9, .... , 9E9] ] ; 

CONST jointtarget P3 : = [[ 0.7, 45.3, 45.3, 

0.0, -44.6, 0.0], [9E9, .... , 9E9] ] ; 



CONST jointtarget P74 : = [[-160.7, 45.2, 0.6, 

16.9, 4.1,-117.0], [9E9, .... , 9E9] ] ; 



! COMMENT Move statements for the RAPID program: 
PROC mainO 

MoveAbsJ PI , v400 , z20 , toolx; 
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MoveAbsJ P2 , v400 , z20 , toolx; 

MoveAbsJ P3 , v400 , z20 , toolx; 

MoveAbsJ P74 , v400 , z20 , toolx; 

MoveL target , vlOO , z20 , toolx; 

ENDPROC 

ENDMODULE 

Executing these robot statements, the ABB control performs only one robot move 
to the target point using the many generated points to perform a „smooth path". 
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Therefore the resulting movement looks much more „natural“, i.e. like a movement of 
a human arm. The reason is, that the programmer usually „divides“ a robot move in 
only a few (typically 3 to 10) intermediate points with a greater distance between 
them, and the robot move looks then more „machine-like“ or „cornered“. A side effect 
of the new method is the generation of moves, which avoid sudden change of speed or 
direction. This leads to a control behavior more avoiding wear and tear of the axis 
motors and gears. Also the generated move was checked to avoid running against axis 
stops. 
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Results of the Implementations for Industrial Robots 

The implementations of GLEAM to industrial robots have been done to demonstrate, 
that GLEAM is able to produce robot code in an industrial environment. The software 
for the ABB robot programming facility generates robot programs with collision free 
robot moves. The collision free path is optimized by criterion’s of different priorities 
selected by the user. E.g. the criterion could be a short Cartesian path of the TCP or a 
short distance of one ore more robot axes. The process of programming optimized 
robot moves can be done by the software tool, which reduces the needed man power 
for the user. 

Another result is the demonstration, that the method GLEAM is applicable to an 
industrial robot control of the user. The advantages are controlling robots without 
complicated mathematics calculations and easy adaptation of different environments 
and robot models. The result can be used for off-line path planning including a multi- 
criteria optimization. The generated move trajectory is an optimization of all 
criterion’s, avoiding problems of defining the configuration space or transfer 
positions. 

The implementation for DaimlerChrysler includes several improvements, e.g. the 
definition of intermediate points by the user. These intermediate points have to be 
passed by the robot trajectory (more or less), the user can influence the robot path and 
help the system to find a collision free path in a shorter time. 

The implementations have been performed on a PC, they allow a low cost solution 
and will be accepted also for smaller companies. The software for the GLEAM 
modules are written in C. The companies ABB and DaimlerChrysler can test the new 
method and decide, if it will be integrated into their programming software tools, the 
implementations are the base of feasibility studies to analyze the new method. 
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Outlook 

The GLEAM concept was realized by implementations for different robots: the 
Mitsubishi RV-M2 robot, the ABB IRB 2400 industrial robot , and the KUKA KR 6 
industrial robot. They generate statements for the robot controls to move the robot on 
an optimized collision free trajectory to a given move target. 

In future an implementation for a professional simulation tool for robots will be of 
interest, e.g. for the IGRIP system of the Deneb company. Such an implementation 
will demonstrate the application of the GLEAM method to a high level language 
simulation command language like GSL, and a complex modeled environment of 
obstacles and robots. The following picture shows the information flow between the 
system components. The user works with the simulation tool as before and models its 
machines, work cell, obstacles, and others. He can program the robot moves in a 
simulation language and simulate the moves. 



User Input to the simulation tool 
Commands 
incl. collision free 
move function 




A new function „optimize moves without collision" will be introduced into 
simulation system. The simulation tool sends the simulation statements already 
programmed by the user to the GLEAM component. GLEAM tries to optimize the 
robot moves (shorter execution time) with respect to collision avoidance. During the 
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optimization process GLEAM sends changed statements in simulation language code 
to the simulation system, which will execute the statements in a „silent“ background 
mode without graphical output. The simulation tool sends back to GLEAM only the 
information about the „fitness“ of the simulated move statements, especially if there 
was a collision or not. After the optimization process GLEAM sends the result to the 
simulation tool: the optimized collision free move statements in the simulation 
language. The user can simulate this result and test, if it is better than his own 
statements. At the end, the simulation tool generates program code for different robot 
control units. 

It is important, that GLEAM can use solutions already programmed by the user as 
an input. GLEAM tries to find a better solution (if there exists one), and it generates a 
trajectory without collisions. The user can visualize the result by simulate it, and he 
can use the generated statements for other programs or to program any robot. 
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Abstract. The use and benefits of self-adaptive mutation operators are well-known 
within evolutionary computing. In this paper we examine the use of self-adaptive 
mutation in Michigan-style Classifier Systems with the aim of improving their 
performance as controllers for autonomous mobile robots. Initially, we implement the 
operator in the ZCS classifier and examine its performance in two animat 
environments. It is shown that, although no significant increase in performance is seen 
over results presented in the literature using a fixed rate of mutation, the operator adapts 
to approximately this v&la-egardless of the initial range. 



1 Introduction 

Within Genetic Algorithms (GAs) [Holland 1975] and Genetic Programming [Koza 
1991] the mutation rate is traditionally a global parameter which is constant over time. 
However, in Evolutionary Strategies [Rechenberg 1973] and later forms of 
Evolutionary Programming (Meta-EP) [Eogel 1992], the mutation rate is a locally 
evolving entity in itself, i.e. it adapts during the search process. This self-adaptive 
form of mutation not only reduces the number of hand-tunable parameters of the 
evolutionary algorithm, it has also been shown to improve performance (e.g. see 
[B ck 1992] for results with a self-adaptive GA). In this paper we examine the use of 
a self-adaptive mutation operator within Michigan-style Classifier Systems (CSs) 
[Holland et al. 1986], more specifically in Wilson’s ZCS [Wilson 1994] system. 

The performance of the new operator within ZCS is examined using two simulated 
autonomous entity/robot - animat [Wilson 1985] - tasks: Woods 1 and Woods 7, 
each of which were originally used by Wilson to introduce and investigate ZCS. In 
both cases it is found that no benefits in performance are found over the results 
presented by Wilson using a fixed mutation rate. However, regardless of the initial 
range for the adapting mutation rates, the final CS animat controllers have roughly the 
same mutation rate as that used by Wilson. That is, we show the principle of self- 
adaptation works within the Classifier System framework. 

The paper is arranged as follows: the next section introduces ZCS. Section 3 describes 
how self-adaptive mutation is implemented and Section 4 describes the tasks and 
examines the effects of the operator. Einally, all results are discussed. 
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2ZCS 

ZCS is a "Zeroth-level" Michigan- style Classifier System without internal memory, 
where the rule-base consists of a number (N) of condition/action rules in which the 
condition is a string of characters from the usual ternary alphabet {0,1,#} and the 
action is represented by a binary string. Associated with each rule is a strength scalar 
which acts as an indication of the perceived utility of that rule within the system. This 
strength of each rule is initialised to a predetermined value termed Sq- 

Reinforcement in ZCS consists of redistributing strength between subsequent "action 
sets", or the matched rules from the previous time step which asserted the chosen 
output or "action". A fixed fraction (P) of the strength of each member of the action 
set ([A]) at each time-step is placed in a "common bucket". A record is kept of the 
previous action set [A].| and if this is not empty then the members of this action set 
each receive an equal share of the contents of the current bucket, once this has been 
reduced by a pre-determined discount factor ("^. If a reward is received from the 
environment then a fixed fraction (P) of this value is distributed evenly amongst the 
members of [A]. Finally, a tax (x) is imposed on all matched rules that do not belong 
to [A] on each time-step in order to encourage exploitation of the stronger classifiers. 
Hence this is different from the traditional "Bucket-brigade" algorithm [Holland et al. 
1986] and is known [Wilson 1994] to be similar to Watkin’s Q-learning [1989] 
reinforcement algorithm. 

ZCS employs two discovery mechanisms, a panmictic GA and a covering operator. 
On each time-step there is a probability p of GA invocation. When called, the GA uses 
roulette wheel selection to determine two parent rules based on strength. Two 
offspring are produced via mutation (probability p) and crossover (single point with 
probability x). The parents then donate half of their strengths to their offspring who 
replace existing members of the rule-base. The deleted rules are chosen using roulette 
wheel selection based on the reciprocal of rule strength. If on some time-step, no rules 
match or all matched rules have a combined strength of less than (|) times the rule-base 
average, then a covering operator is invoked. 

The default parameters presented for ZCS, and unless otherwise stated for this paper, 
are: N = 400, Sq= 20, P = 0.2, y = 0.71, x = 0.1, x = 0.5, p = 0.002, p = 0.25, ^ = 0.5 

Thus ZCS represents a "basic classifier system for reinforcement learning that retains 
much of Holland’s original framework while simplifying it so as to increase 
understandability and performance" [Wilson 1994]. For this reason the ZCS 
architecture has been chosen to examine the basic behaviour of classifier systems with 
self-adaptive mutation rates. The reader is referred to [Wilson 1994] for full details of 
ZCS. 
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3 Self-Adaptive Classifier System Controllers 

3.1 Self-Adaptation 

In this paper we use the same form of self-adaptive mutation as in Meta-EP. That is, 
each rule has its own mutation rate p, stored as a real number, which is passed to its 
offspring, either under recombination or directly (depending upon the satisfaction of 
X). The offspring then applies its mutation rate to itself using a Gaussian distribution, 
i.e. pj’ = Pi + N(0,pj), before mutating the rest of the rule at the resulting rate. It is 
noted that this form of self-adaptation is simpler than that typically used in 
Evolutionary Strategies, where a Lognormal is applied to p, however the simpler form 
is shown to be adequate here and has been suggested to work better in noisy 
environments [Angeline et al. 1996]. 

We also note that this is in contrast to the adaptive form of crossover introduced by 
Wilson (1987) for CS, under which a system entropy measure was used to alter the 
operator rate; Wilson showed benefits from increasing crossover as entropy dropped 
using predetermined rules of change. 

3.2 Classifier Systems in Evolutionary Robotics 

A number of investigators have examined the use of Classifier Systems in 
evolutionary robotics. Dorigo, in conjunction with many others (see [Dorigo & 
Colombetti 1999] for a comprehensive overview), has used multiple CSs in a 
hierarchy to control an autonomous robot in a variety of environments. To our 
knowledge this remains the only hardware implementation to date. A large body of 
work exists on the use of CSs to control simulated robots however, e.g. [Riolo 1991], 
[Cliff & Bullock 1993], [Donnart & Meyer 1994], [Stolzmann 1999], etc. The reader 
is referred to [Lanzi et al. 2000] for a full CS bibliography. 

The performance of a Michigan-style classifier system - ZCS - with self-adaptive 
mutation in simulated evolutionary robotics tasks is now examined, with the aim of 
determining ways to improve their use in real environments. 

4 Results in Woods 1 and Woods 7 

4.1 The Tasks 

Wilson [1994] introduced two multi-step "woods" environments with which to 
examine the performance of ZCS. Woods 1 is a two dimensional rectilinear 5x5 
toroidal grid. Sixteen cells are blank, eight contain rocks and one contains food. ZCS 
is used to develop the controller of a robot/animat which must traverse the map in 
search of food. It is positioned randomly in one of the blank cells and can move into 
any one of the surrounding eight cells on each discrete time step, unless occupied by 
a rock. If the animat moves into the food cell the system receives a reward from the 
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environment (1000), and the task is reset, i.e. food is replaced and the animat randomly 
relocated (Figure 1). 



On each time step the animat receives a sensory message which describes the eight 
surrounding cells. The message is encoded as a 16-bit binary string with two bits 
representing each cardinal direction. A blank cell is represented by 00, food (F) by 11 
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Fig. 1 The Woods 1 
environment showing the 
CS controlled animat * 
adjacent to the food goal. 



and rocks (O) by 10 (01 has no meaning). The message is ordered with the cell directly 
above the animat represented by the first bit-pair, and then proceeding clockwise 
around the animat. 



The trial is repeated 10,000 times and a record is kept of a moving average (over the 
previous 50 trials) of how many steps it takes for the animat to move into a food cell 
on each trial. If it moved randomly Wilson calculates performance at 27 steps per trial, 
whilst the optimum is said to be 1.7 steps. 

Woods 7 is a more complex and non-Markov version of Woods 1 on a 58x18 grid. 
Fifty-seven cells evenly scattered around the map contain food. Each of these has 
rocks positioned randomly in two of the eight surrounding cells. The rest of the map 
is blank. Wilson states that random search will take 41 steps to food, whilst the 
optimum is 2.2 steps per trial (not shown - see [Wilson 1994]). It is noted that, since 
ZCS has no temporary memory (see [Cliff & Ross 1995][Tomlinson & Bull 1998]), it 
cannot be expected to solve Woods 7 optimally, however it is still of interest to see 
how well [it] can do [Wilson 1994]. All inputs and other task details are the same in 
Woods 7 as Woods 1. 



All results in this paper are the average of ten runs. 

4.2 Results 



Figure 2 shows the performance of the self-adaptive mutation operator within ZCS on 
Woods 1. Flere the initial possible range of mutation rates was centred around the 
fixed rate of 0.002 used by Wilson, i.qa<flM)04. It can be seen that the use of self- 
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adaptation has had no real beneficial/detrimental effects on performance, if anything 
learning is a little quicker (Fig. 2a). Examination of the average mutation rate in the 




rule-base (Fig. 2b) shows that it rises slightly from the mean of 0.0020 up to around 
0.0025. That is, although no significant improvements in performance are seen here, 
there is obviously a slight selective pressure for a higher mutation rate than that used 
by Wilson. 

The resulting rule-bases at the end of the 10,000 trials have been examined. Although 
the average mutation rate appears roughly equal to the fixed rate used by Wilson, 
analysis shows a wide range of individual rates. Typically, a large proportion of the 





Fig. 3 Showing the effects of a higher initial mutation rate seed in Woods 1. 

rules have no mutation, i.pj = 0, whilst others have mutation rates up to and over 1. 
The former of these is associated with rules for cells closest to the food goal in the 
environment, whilst the latter are associated with rules for cells furthest from the goal. 
Hence it appears that the degree of evolutionary search on the rules of a particular 
situation/cell, during the learning process, is directly correlated with the position of 
the rules in the inductive/reinforcement chaiRuns over a much larger number of 
trials show the average mutation rate eventually going to zero (not shown). 
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We have also examined the robustness of the self-adaptive approach to starting the 
system with an inappropriate mean mutation rate. That is, we were interested in 
whether self-adaptation can be used to remove the mutation parameter from the 



(b 

designer’s control somewhat and hence ease the use of Classifiers in complex tasks, 
particularly evolutionary robotics. Figure 3 shows results from an initial range 
0<|4<0.5. It can again be seen that no real benefit/detriment is found over the use of a 
fixed mutation rate (Fig. 3a), but that now the average mutation rate continually falls 
towards 0.002 after a slight delay (Fig. 3b). 

Results from the more complex and non-Markov Woods 7 task were very similar to 
those above. Figure 4 shows that there is no significant change in performance when 




Pfoblems Probl«ms 



(a) 

Fig. 5 Showing the effects of a higher initial mutation rate seed in Woods 7. 

the initial mutation rates are seeded around Wilson’s fixed rate of 0.002 (Fig. 4a). 
There is a rapid increase in the average mutation rate, up to around 0.004 (Fig. 4b), 
which then deviates around 0.003, a slightly higher rate than seen in Woods 1. That is, 
in a different environment we see a different form of self-adaptation occurring in the 
mutation rate. Tests for robustness with much higher initial mutation rates (e.g. 
0<|4<0.5) also showed no change in performance and that the average mutation rate 
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falls toward 0.002, although more quickly than in Woods 1 (Figure 5). 

Due to the complexity of the environment analysis of the resulting rule-bases is more 
difficult here, but the same general correlated effect in terms of mutation/strength 
convergence appears to occur as described above in Woods 1. 

5 Conclusions 

In this paper it has been shown that it is possible to use a self-adaptive mutation 
operator within Michigan-style Classifier Systems - specifically Wilson’s ZCS - with 
the aim of improving their performance as controllers for autonomous mobile robots. 
We are now moving these experiments onto a real robot platform in conjunction with 
the Intelligent Autonomous Systems Laboratory, Faculty of Engineering at UWE. 
Further enhancements to the self-adaptive mechanism are also currently being 
investigated, as well as implementing them in the more sophisticated XCS system 
[Wilson 1995]. 
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Abstract. A hybrid approach for learning reactive behaviours is pre- 
sented in this work. This approach is based on combining evolutionary al- 
gorithms (EAs) with the A* algorithm. Such combination is done within 
the framework of Dynastically Optimal Forma Recombination, and tries 
to exploit the positive features of EAs and A* (e.g., implicit parallelism, 
accuracy and use of domain knowledge) while avoiding their potential 
drawbacks (e.g., premature convergence and combinatorial explosion). 
The resulting hybrid algorithm is shown to provide better results, both 
in terms of quality and in terms of generalisation. 



1 Introduction 

The control of autonomous mobile agents is a complex task to which great efforts 
are devoted due to its practical applications. In general, such control is achieved 
by means of both planning and reactive components [13]. Each of these compo- 
nents has it own particularities, and can be examined in combination (e.g., [1, 
2,7]) or in isolation (e.g., [11,12]). In line with the latter, this work focuses on 
the acquisition of reactive behaviours in mobile agents. 

Reactive behaviours are driven by a stimulus-to-response mapping, i.e., the 
agent receives some information about its local environment and decides the 
action(s) to carry on exclusively on the basis of such information. This kind of 
behaviour has usually the advantage of not requiring any underlying global model 
of the world in which the agent is located. The obvious drawback of reactive 
systems is the fact that they can get stuck into dead-ends, situations in which 
the correct action does not only depend on the locally available information but 
also the structure of the world at a higher-level (hence the necessity of long- 
term planning capabilities). Nevertheless, reactive systems have been shown to 
provide a very good performance in a wide variety of scenarios and remain a 
very suitable option when response-time is critical. 

There exist several techniques for designing reactive systems. These can be 
typically classified into reinforcement learning and optimisation techniques. Al- 
gorithms such as Holland’s bucket brigade, Sutton’s temporal difference learning 
or Watkins’s Q-learning lie in the first class. Within the second class, evolution- 
ary algorithms deserve special attention because of their power and flexibility. 
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Regarding the use of these techniques for this purpose (e.g., [8,9, 11]), a criti- 
cal point is the use of as much domain knowledge as possible. Otherwise, the 
user would be relying on a fortuitous matching between her algorithm and the 
problem under consideration [14] . Such specialised algorithms are usually termed 
hybrid evolutionary algorithms [6]. 

This work presents a hybrid evolutionary algorithm for acquiring reactive 
behaviours. In the proposed algorithm, domain knowledge is included by using a 
specialised technique (the A* algorithm) as an internal operator. The remainder 
of the article is organised as follows. First, the agent and the worlds used in the 
experiments are described (Sect. 2). Next, the classical A* approach for solving 
the posed problem is shown (Sect. 3). Then, the hybrid algorithm is introduced 
(Sect. 4). Subsequently, experimental results are presented (Sect. 5). Finally, 
some conclusions are extracted and future work is outlined (Sect. 6). 



2 The Agent and its World 

The agent used in this work is located in a two-dimensional toroidal grid-world 
in which several obstacles are distributed. The purpose of the agent is to reach 
a certain target point from its initial location within an allowed time. To do so, 
the agent is capable of making some elementary actions such as moving straight 
ahead a single grid square, turning 90° to its left, or turning 90° to its right. 
Obviously, the agent must avoid obstacles while navigating through its world. 
For this purpose, it is equipped with proximate sensors that can inform of the 
presence or absence of obstacles in front of the agent, 90° to its left, or 90° to 
its right (see Fig. 1, left). In addition, these sensors can also detect whether the 
target point is in any of these three locations or not. 




Fig. 1. (Left) Structure of the agent used in experiments. (Right) Example world and 
regions into which it is divided according to the location of the target point. 
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The agent is equipped with a direction sensor as well. This sensor allows 
determining in which of four imaginary regions of the world the target point is 
located. These regions are illustrated in Fig. 1 (right). It must be noted that 
these regions are not absolute but relative to the agent’s actual orientation. For 
example, the agent is facing North in Fig. 1 and hence the target point is in zone 
1. Now, if the agent turned 90° to its right, the target would be in zone 0. Notice 
also that these regions are determined taking into account the toroidal shape of 
the world. Thus, if the agent were a few positions South from the location shown 
in the previous example, the target point might happen to be in zone 2. 

According to this description, the goal is to design a reactive behaviour allow- 
ing the agent reaching its target in as many situations as possible. Such reactive 
behaviour can be defined in a variety of ways, e.g., using a neural network [15], 
a fuzzy rule-base [8], a cellular automata [3], etc. This work is in line with the 
latter approach. To be precise, a lookup-table is sought relating every possible 
sensorial input with a primitive action. At each time-step, the agent must look 
up the action that corresponds to the current inputs and carry it out. Since each 
proximate sensor can provide three different inputs (OBSTACLE, NO-OBSTACLE, 
TARGET), and the direction sensor can return four values, the resulting table has 
3^ • 4 = 108 entries. Since three primitive actions - MOVE-AHEAD, TURN-LEFT, 
TURN-RIGHT - are available, this implies a search space of 3^°® 3 • 10®^ tables. 

3 A Classical Approach: A* 

A classical approach for finding the lookup-table mentioned above is the utili- 
sation of the A* algorithm. Based on incrementally constructing solutions in an 
intelligent fashion, this technique constitutes a powerful tool for solving search 
problems to optimality. Before getting into the application of this technique to 
the design of reactive behaviours, some notation details must be given. 

Let >V be the current world, and let a be the configuration of the agent 
(position and orientation). Now, let I{W,a) be the sensorial input of the agent 
when configured according to a. Let M be a (possibly underspecified) function 
relating sensorial inputs with actions, and let M\ D M 2 whenever M 2 provides 
the same outputs that M\ does and M 2 is defined in at least one case in which 
Ml is not. Finally, let r be the maximum allowed time for reaching the target 
and let S be a function such that S(W, a,M) = ((ai, ■ ■ ■ ,at),r]). This function 
provides a trace of the agent trajectory across configuration space when behaving 
according to M. The value rj is an indication of the final status of the agent: 
AT-TARGET, COLLISION, TIMED-OUT or UNKNOWN. This latter value is returned 
whenever no action is specified in M for the current input. 

Now, the application of the A* algorithm requires the availability of an op- 
timistic evaluation function ^ such that tp{W,a,M) provides a lower bound on 
the number of steps necessary for reaching the target when the agent is con- 
figured as a and behaves according to any M ' , M D M' . It is easy to see that 
making ^ return the Manhattan distance from the agent’s current location to 
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the target point fulfils this requirement. Having defined this function, the whole 
process is as follows: 

1. Let Pq = (cto, Mq, 0, to), where cto is the initial configuration of the agent, Mq 

is a fully underspecified function, and to = cto, .^o)- Let P* = (Mo, oo) 

be the current best solution. Insert Pq in the node queue. 

2. If the node queue is empty, go to 3. Otherwise let Pj = (a,M,h,t) be the 
first element in the queue. 

(a) Let ~(W,a,M) = {{ai, - ■ ■ ,ak),r]). 

(b) iir]^ UNKNOWN then 

i. if r] = COLLISION or r] = TIMED-OUT then P = {M ,t' + r), where 
t' = h + k + il)(yV,ak, M). 

ii. if r] = AT-TARGET then P = (M, h + k). 

hi. If P is better than P* , update the latter and purge nodes in the 
queue, 
iv. Go to 2. 

(c) Create three nodes Pj^^ , and Pj^^ from Pj. Each node is 

= (o:k,Mr,h',tr), where Mr is obtained by extending M to re- 
turn the rth possible action when the input is P(>V, at), h' = h + k, and 
tr = 'tp{W,ak,Mr). Insert these nodes in the queue keeping it ordered 
according to the sum of the last two components of each node. 

(d) Go to 2. 

3. Return P* 

This algorithm will thus return the lookup-table allowing the agent reach the 
target in minimal time from the given starting point. Since the problem has been 
posed with the goal of obtaining a generalisable reactive behaviour, the process 
must be slightly modified. To be precise, a training set is selected and the A* 
algorithm tries to find the table that minimises the sum of the times required 
to reach the target in each training case or, if such a solution is not possible, 
a table that firstly maximises the number of training cases solved and secondly 
minimises the total time. Notice that no global model of the world (i.e., high- 
level knowledge about the distribution of obstacles) is required. All information 
used for finding the optimal solution is locally obtained through simulation. 

This algorithm has been evaluated on a set of nine different worlds. These 
worlds are named as Wa;y, where x € {10,25,50} indicates the dimension of 
the world (each world is a a; x a; grid), and y € {a, b, c) indicates the density of 
obstacles (5%, 10% and 20% respectively). For each world, a training set of five 
cases has been selected. Subsequently, the best solution found has been tested 
for generality on a test set whose size depends on the dimension of the world 
(50, 400 and 2000 cases respectively). The results are shown in Table 1. 

These results are very indicative of the two main drawbacks of the A* algo- 
rithm. On the one hand, it is very sensitive to the size of the task to be solved. 
As it can be seen, the algorithm expended a high computational effort for solving 
W25b and ran out of memory in three cases (W25c, W50b, and W50c). More- 
over, it did not find any fully satisfactory solution for all training cases in W50b 
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Table 1. Results of the A* algorithm on nine different worlds. The cost values are 
measured as the number of single simulation steps carried out. 



World 


Timeout 


Optimal solution 


Iterations 


Cost 


Performance 
on test set 


WlOa 




13.20 


11615 


289857 


74% 


WlOb 


25 


13.80 


19664 


300391 


88% 


WlOc 




14.00 


26429 


281091 


58% 


W25a 




32.40 


75818 


10352732 


63% 


W25b 


150 


31.60 


222788 


29083614 


48% 


W25c 




[26.40, 51.00] 


>300000 


>25000000 


36% 


W50a 




54.60 


119864 


36861614 


66% 


W50b 


400 


[45.60, 479.40] 


>150000 


>37000000 


14% 


W50c 




[34.80, 1629.80] 


>110000 


>16000000 


2% 



and W50c (1 and 4 training cases were left unsolved^). On the other hand, the 
solutions found are not very generalisable. This is a direct consequence of the 
internal functioning of the A* algorithm. Assume that the final solution is found 
when evaluating node Pj. This node was obtained as successive extensions of 

■ ■ ■ > -Fq • Hence, it contains information regarding the best decisions 
to be taken only in the situations found during this optimal path, i.e., the path 
from the root node of the implicitly defined search tree to the optimal leaf node. 
All that may have been learnt in solving other situations is discarded since these 
situations do not take place in this optimal path. 




2 3 

Number of Irainirtg cases 



Fig. 2. Growth of the computational cost of the A* algorithm when the number of 
training cases is increased. 

^ These results were not bad a priori since there might exist no better solution. How- 
ever, further experimentation with the hybrid EA showed that this was not the case. 
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This generalisation problem could be solved by considering a larger training 
set whose optimal solution covered all possible situations. However, the subse- 
quent combinatorial explosion makes this approach unrealistic. This is illustrated 
in Fig. 2. As it can be seen, the computational cost of the algorithm grows very 
fast when the size of the training set is increased. For this reason, it is clear that 
alternative approaches must be found. These will be discussed in next section. 

4 The Hybrid EA-A* Approach 

Evolutionary algorithms constitute a very suitable alternative to A* for finding 
the lookup-table. A naive approach for applying EAs to this problem would 
firstly consist of defining an encoding function for storing the lookup-table into 
an individual, e.g., a linear chromosome in which the rows of the table are 
consecutively arranged. Since this is an orthogonal representation [10] (i.e., all 
combinations of genes are feasible) , the next step would simply involve selecting 
any of the standard genetic operators that can be found in the literature (e.g., 
single-point crossover - SPX -, uniform crossover - UX -, etc.). 

However, such a simple approach is likely to provide very poor results. Recall 
that this is highly epistatic problem in which the value of each gene (i.e., a spe- 
cific action to be carried out when a certain sensorial input is received) does not 
contribute with a fixed amount to the fitness of an individual. On the contrary, 
the goodness of the reactive behaviour defined is determined by the interplay 
between all genes. Eor this reason, a blind recombination operator that randomly 
shuffles the genetic material of recombined solutions will provably produce solu- 
tions with a phenotype (reactive behaviour) completely unrelated to the parents, 
even when the latter are genotypically similar. In an extreme situation, it may 
even reduce to macromutation. 

The algorithm would be largely more effective if it were able to extract pos- 
itive behavioural patterns from existing solutions and transmit them to the off- 
spring. This can be achieved within the framework of Dynastically Optimal 
Eorma Recombination [4] (DOR). This framework comprises a family of recom- 
bination operators of the form 

DOR : 5 X 5 X 5 [0,1], (1) 

where S is the search space and DOR(a;,y, 2 :) is the probability of generating 
2 : when recombining x and y. Besides the obvious DOR(a;, y, 2 ) = 1, the 

probability distribution induced by these operators verify that 



D0R(a;,y,2) > 0 => {[2 G r{{x,y})] A [Vw G r{{x,y}) : (f>{w) > (f>{z)]} (2) 

where (f) is the fitness function (to be minimised without loss of generality) and 
r{{x,y}) is the dynastic potential [10] of x and y, i.e., the set of solutions that 
can be built using nothing but the information contained in x and y. 

Thus, the solutions created by DOR are the best that can be constructed 
using the genetic material of the parents. On the one hand, this implies that 
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DOR is a fully transmitting operator, i.e., no implicit mutation (genetic infor- 
mation not present in any of the parents) is introduced in the offspring. On the 
other hand, the fitness-oriented functioning of DOR makes valuable portions of 
solutions be transmitted to offspring only if they contribute to a good resulting 
behaviour. In other words, DOR is capable of identifying valuable high-order 
formae, preventing their disruption. This intelligent combination of information 
has provided very good results on epistatic problems [5]. 

In order to implement DOR, it is required to use an embedded A*-like mech- 
anism so as to find the best solution in the dynastic potential of the parents. In 
this case, the algorithm described in Sect. 3 can be used. It is only necessary to 
modify step 2c by considering that the possible actions to be taken in a given 
situation T are just those present in any of the parents for T. Notice that the 
search carried out by this subordinate A* algorithm is thus restricted to small 
portions of the search space and hence its computational cost is largely reduced 
with respect to the original unrestricted version. Moreover, individuals in the 
population tend to be more similar as the EA converges and, subsequently, the 
dynastic potential of selected solutions tends to be smaller and DOR is less 
computationally expensive. 

This combination of EAs and A* has an additional advantage. Each individ- 
ual carries an information that reflects its past evolution (in fact, the evolution 
of its ancestors). This way, things that were learnt in the past are retained as 
long as they do not negatively affect the present behaviour. This “accumulated 
history” effect is also present in a simple EA, but the learning capabilities of the 
hybrid algorithm are larger. Eor this reason, solutions obtained with the hybrid 
EA are expected to be more general than either the EA or the A* algorithm by 
themselves. This will be studied in next section. 



5 Experimental Results 

Experiments have been done with a steady-state EA {popsize = 100, Pc = .9, 
Pm=l/chromosomeLength) using ranking selection = 2.0, rj~ = 0.0). This 
algorithm has been run 40 times for each operator and test world. In order to 
make a fair comparison between DOR and the other simpler operators, each run 
is terminated when a fixed number of simulation steps (r • 10® in these experi- 
ments, where r is the timeout value) is reached. Thus, the internal calculations 
performed by DOR are effectively accounted. As in Sect. 3, a training set of five 
cases is used in the fitness function. 

Eirst of all, Eig. 3 shows how the hybrid EA is much more successful in solving 
the training cases. As it can be seen, while standard operators only provide an 
acceptable performance on the smallest instances and with the lowest obstacle 
density, DOR consistently yields satisfactory results: above a 70% of the runs 
provide a fully successful solution for the training set (the percentage is 100% 
for 5 out of 9 test worlds). The exception is world W50c for which none of the 
operators could find a full solution (it must be noted that such a solution may 
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Fig. 3. Number of runs in which each operator provided a fully satisfactory solution 
for the training set. 



not exist). Nevertheless, DOR was capable of solving 3 out of the 5 training 
cases while SPX could only solve one and UX could not solve any of them. 

Table 2 shows a more detailed summary of the results. Notice that DOR 
is not only more effective in finding satisfactory solutions, but also provides 
higher-quality results. By comparing the median values^ provided by DOR with 
the optimal/best-known solutions (see Table 1), it can be seen that DOR yields 
near-optimal solutions. Moreover, the lower variance of DOR results with respect 
to SPX and UX indicates a more stable algorithm. 



Table 2. Comparison of different genetic operators on nine different environments. All 
results correspond to series of forty runs. 



World 


Timeout 


SPX 


UX 


DOR 1 


mean 


a 


median 


mean 


a 


median 


mean 


a 


median | 


WlOa 




17.36 


12.06 


16.00 


15.88 


7.47 






0.87 


m 


WlOb 


25 


22.15 


25.96 


16.20 


18.10 


9.26 




mm 


0.74 


gil 


WlOc 




51.49 


38.39 


59.60 


53.85 


38.81 


gel 


18.75 


10.75 


14.00 


W25a 




176.70 


210.00 


45.60 


121.63 


193.32 


40.00 


33.17 


2.14 


32.40 


W25b 


150 


376.08 


285.43 


399.30 


380.96 


272.08 


468.40 


36.65 


6.67 


34.00 


W25c 




520.32 


179.07 


616.80 


502.23 


218.40 


616.40 


99.92 


107.70 


38.80 


W50a 




442.39 


348.27 


438.00 


423.87 


335.45 


438.00 


61.17 


11.72 


58.80 


W50b 


400 


1532.52 


554.56 


1636.60 


457.65 


457.65 


1636.60 


337.32 


506.03 


103.8 


W50c 




2021.45 


60.77 


2031.20 


2031.20 


0.00 


2031.20 


1638.78 


538.75 


1650.60 



^ The median value seems to be a more representative measure of the quality of the 
results than the mean value since the former is much less sensitive to outliers. Fur- 
thermore, it provides an reasonable alternative to averaging the fitness of solutions 
that solve the whole training set with solutions that do not solve any training case. 
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Fig. 4. Percentage of the test set solved for each of the techniques considered. 



Finally, the results obtained with the EA are tested for generality. Fig. 4 
shows the results. Firstly, notice the poor results of standard EAs. The solutions 
provided by UX and SPX do not reach 50% success in 6 out of 9 worlds. The 
A* algorithm performs better than standard EAs, but its performance quickly 
drops when the density of obstacles is increased. The hybrid EA provide the 
overall best results, outperforming both A* and standard EAs on all worlds. 
Moreover, this improvement is larger on instances with higher obstacle densities. 
It must be noted that the results on W50c are not satisfactory for any algorithm 
(although the hybrid algorithm remains the best) . This is a really hard instance 
as mentioned before, and may require longer evolution times and/or a larger 
training set to cope with such a tough environment. 



6 Conclusions 

This work has presented a hybrid approach for learning reactive rule-bases. By 
combining EAs with the A* algorithm, a synergetic system has been achieved. 
This hybrid algorithm has been shown to provide higher-quality results than 
standard EAs. These results are also better than those of the A* algorithm in 
terms of their generalisation to previously unseen test cases. Furthermore, the 
hybrid EA is capable of tackling instances in which the A* algorithm would 
suffer the effects of the combinatorial explosion. 

Future work will try to extend these results to more sophisticated agents. 
In this sense, notice that most details of the agent are encapsulated within the 
simulation function S and hence they do not affect the presented algorithm 
qualitatively. Nevertheless, it is clear that issues regarding simulations of higher 
computational cost are worth studying. Work is in progress in this area. Addi- 
tionally, new environments and tasks to be solved will be tackled as well. 
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Supervised Evolutionary Methods in 
Aerodynamic Design Optimisation 
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Abstract. This paper outlines the application of evolutionary search 
methods to problems in aeronautical design optimisation. The proce- 
dures described are based on the genetic algorithm (GA) and may be 
applied to other areas. Although easy to implement, a simple genetic al- 
gorithm is often found in applications to be of low eflfciency and to suffer 
from premature convergence. To improve performance, two alternative 
strategies are investigated. In the hrst, a learning classiher scheme is 
used to tune the GA for a particular class of problems. The second strat- 
egy uses a parallel distributed genetic algorithm supervised by single or 
competing agents. The implementation of each procedure, and results for 
typical design problems are outlined. The agent supervised distributed 
genetic algorithm is found to provide a model with a very high degree of 
adaptibility, and to lead to considerably improved eflfciency. 



1 Introduction 

The principles of evolutionary computation are well established and are described 
in texts such as [1, 3, 2, 4]. These methods continue to grow in diversity, and 
are becoming more commonly used for many problems in engineering design. In 
aerospace vehicle design, the genetic algorithm (GA) has been applied to a range 
of problems, as described for example in [5, 6, 7, 8, 9]. The apparent robustness 
of the GA, the ease with which it can be applied, and its ability to handle 
discontinuous or even discrete data make it attractive as a search procedure. 

There are other search procedures however (including various gradient search 
methods) which are more efficient than the GA, albeit that they may only 
work for a more restricted class of problems. Furthermore, although evolutionary 
methods have been found to be quite adept at locating global optima in highly 
multimodal problems, there is usually no guarantee that they will, and their 
rate of convergence to the optimum solution can be very slow. Ideally one would 
like an optimisation procedure to work well across a broad range of problems. 
Bearing in mind the no free lunch ‘rule’ however, some degree of matching of 
technique to problem seems inevitable. 

Three simple ways of improving an evolutionary search procedure for a par- 
ticular type of application are: 

1. tuning the parameters to suit the class of problem, 

2. improving its ability to adapt to the problem, 

3. hybridisation with other techniques appropriate to the class of problem. 

S. Cagnoni et al. (Eds.): EvoWorkshops 2000, LNCS 1803, pp. 357-366, 2000. 
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We discuss only the first two of these here, and will use the GA as our basic 
evolutionary procedure. The construction of a suitable hybrid is very problem 
dependent, and we consider that an improved GA may either outperform a 
hybrid routine or may at worst lead to more effective hybrids. (Hybrid routines 
will in any case be discussed elsewhere). The rest of the paper is organised as 
follows. The common design problem of shape or form optimisation is introduced 
in the context of aerodynamic or aeroelastic design. The application of a real 
encoded GA to a typical problem is also briefly outlined. After this, we examine 
the use of a learning classiher system to tune the GA parameters, for a particular 
class of such design problems. The distributed genetic algorithm or DGA is then 
introduced, and hnally the use of agent and multi-agent supervision to improve 
the adaptibility of the DGA is described. 

2 Shape optimisation &z. outline of basic GA 

A frequent task in aeronautical design is to hnd the ‘best’ aerodynamic shape 
of an airfoil (in 2D) or a wing (in 3D), subject to certain constraints. The basic 
GA we use for this task is constructed as follows. 

— The encoding specihes the shape of a trial solution, though it may also spec- 
ify structural parameters such as type of material, weight, rigidity etc. For 
simplicity, let us consider the case of a 2D wing section or airfoil. Then an 
array c[j] may be used to specify the ordinates of a B-spline control polygon, 
which in turn describes an airfoil shape, as shown in hg. 1. 

We use a real array of 20 control point ordinates to encode the shape. (For 






( Chromosome 
f Population 



Chromosome array: ,Y„) 




Fig. 1. Representation of airfoil geometry. 



aeroelastic optimisation of wings instead [13], we encode the design as a sur- 
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face interpolated between a series of spanwise sections, with the encoding 
specifying a standard basic prohle, together with the chord, thickness, twist 
and structural parameters for each section). 

— An imtial population of random airfoils is generated, (each initial shape is 
ensured to be valid, i.e. encodings which produce surface crossings or violate 
thickness constraints are discarded). 

— The fitness is evaluated using a CFD flow solver (or a CFD + structural 
solver). (Either an unstructured mesh based solution of the Euler equations, 
or a viscous/inviscid panel flow solver is used, [11]. In a direct optimisation, 
the search is for the shape which best meets the requirements (e.g. high 
lift/drag (L/D)), whereas in an inverse optimisation, the shape which best 
matches a given pressure coefhcient distribution (Cp) is sought. 

— Fitness values are range-scaled and remapped to the interval [0,1], with 
individuals below the mean htness assigned a base value (typically 0(0.4)), 
and those above the mean assigned a value scaled quadratically up to 1. 

— Roulette wheel is used for selection in the basic GA and DGA operations, 
though the learning GA allows other methods (binary tournament etc.) 

— Two point crossover is again the basic crossover type, (with one point, uni- 
form etc. allowed in the learning GA). Mutation and crossover are applied 
as separate operators. The probability of mutation of a given gene is low, 
(typically 0.005), but is commonly increased at later stages. 

— Elitist population replacement is applied, where the best of each genera- 
tion is automatically carried through, (here together with a slightly mutated 
copy of the best). The population replacement routine ensures that no over- 
replication (i.e. excess identical or almost identical duplication) occurs. 



3 Classifier Learning Directed GA 

The classiher learning system is described in standard texts on machine intelli- 
gence; much of the research in this area also follows from ideas put forward by 
Holland [14]. The classiher learning system adds a layer on top of the GA. Rules 
to control operators (mutation/crossover, type of crossover, selection scheme 
etc.), and parameter values (e.g. mutation rate) are prescribed, and their ef- 
fectiveness when implemented in the GA are assessed. In a static mode, the 
entire set of rules can undergo genetic operations to evolve better rules and rule 
combinations. In a dynamic mode, a system of reward paybacks can be used to 
determine rule selection. 

The objective of the procedure here was to train the GA in a static mode 
for a particular problem type. The efficiency of the trained and basic GAs were 
then compared; hrstly when given problems of a very similar nature to those 
used in training, and secondly for slightly different problems. In the learning 
GA outlined in hg. 2, an initial population of 30 rules was used, which were 
randomly initialised. The rules were encoded as 15 genes in an IF-THEN-AND 
conhguration, and were designed to work in groups, with the hnal 2 genes being 
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Fig. 2. Classifier Learning System/GA 



index references to other rules to be used in combination with the current one. 
The rule gene string comprised: 

— two IF operators, with parameter values determining the generation number, 
and average gene diversity respectively which triggered operation of the rule, 

— the following 11 gene values (type THEN) set operator type and parameters, 

— the hnal two (type AND) determined the combination of rules to be used. 

The procedure was repetitively applied to the inverse design of respectively 
symmetric and non-symmetric airfoils. (To recall, in inverse design, the problem 
is to hud the shape which matches a specihed pressure coefhcient {Cp) distribu- 
tion; it is often required in real applications, and for comparative tests of search 
methods purposes it is preferrable to direct design, as the target is given). Each 
rule was tested twice, and the top 10 peforming rules were then isolated and 
repetitively tested over 15 separate runs. 

The G A was trained for inverse optimisation of symmetric and non-symmetric 
airfoil sections, (respectively using NACA airfoil types -0012, -0022, -0024for sym- 
metric training, and -23015, -4421 for non-symmetric training). Figure 3 shows 
results for the inverse design of a non-symmetric airfoil (23015), comparing the 
performance of the GA using: rules developed for this type of airfoil, rules devel- 
oped for a different (symmetric type), and the basic (untrained) GA. It can be 
seen that the trained GA outperforms the baseline GA for inverse optimisation 
whether it is trained on a class of similar airfoils, or a dissimilar class. How- 
ever, the difference in performance resulting from the class of problem used for 
training shows how very specihc training may be needed to obtain the highest 
gains. 

At present, the cost of GFD evaluations is generally so high, that the ben- 
ehts of training do not appear worthwhile. However other possibilities, such as 
using a simpler approximate evaluation method for training only, or reducing 
the complexity of the scheme, (and hence degree of training), may yet render 
it more practical in this area. Also, the classiher system may still prove useful 
within the context of an agent supervised DGA. 
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Fig. 3. Application of learning GA to inverse design of NACA-23015 airfoil. Compar- 
ison of performance with rule sets developed for similar airfoils, for different airfoils, 
and basic GA performance. 



4 Distributed Genetic Algorithm (DGA) 

Previous work has already shown that the DGA outperforms the GA on many 
test problems and in design optimisation, [10], [16]. Applications of the DGA 
to aeronautical design problems are also described in [11, 13]. Briefly the DGA 
differs from the standard GA in that the population of trial solutions is split into 
semi-isolated subpopulations or ‘demes’. (The demes are considered analogous to 
island populations, where geography acts as a barrier to exchange). Restrictions 
on the recombination and genetic exchange between subpopulations are imposed; 
the exchange is limited to the migration of a few individuals (often only the best 
one or two) from one neighbouring deme to another every m generations, with 
typically m = 5 or m = 10 in our implementations. 

Thus the parameter set for the basic GA (operator probabilities, selection 
mode, etc.) is enlarged to include the number of individuals migrating, barriers 
to acceptance of immigrants, exchange frequency, geographical exchange radius 
and topology of the demes. With limited exchanges between demes, the DGA is 
then ideally suited for coarse grain parallelisation (hg.4) whether on a parallel 
supercomputer or network of workstations. Nang [12] surveys the parallel GA, 
of which the ‘stepping stone’ connected DGA shown in hg. 4 is one type. 

For a workstation network [11], provided the flow solution can be run on a 
single workstation, the only communication required between processors involves 
the exchange of a limited number of chromosomes, at intervals of several gener- 
ations. The communication requirements are extremeley low, given that by far 
the bulk of the computational effort is devoted to the flow solution which per- 
forms the evaluation. Each processor may be responsible for a number of demes. 
For a heterogeneous network of processors of different speeds or loadings, load 
balancing can be achieved by varying the mapping of demes to processors, or by 
altering the number of individuals treated by a given processor. 
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Fig. 4. Distributed GA (DGA) mapped to processor array. 



4.1 Application to aeronautical design 



Inverse design Comparison of the convergence behaviour obtained using a dis- 
tributed [13] and a conventional GA, with the same total population and number 
of evaluations, for the viscous inverse design of a NLF(1)-0115 airfoil is shown in 
hg. 5. For the DGA, the population of 180 was distributed onto 9 subpopulations 
of 20, each residing on a different processor, as in hg.4 above. The migration be- 
tween the islands occurred in a stepping stone fashion, ie. migration occurred 
only between immediate neighbours every hve generations. As the results indi- 
cate, the DGA greatly outperforms the conventional GA algorithmically. This 
gain is then further multiplied almost ninefold with the distributed processor 
implementation. (The implementation was done using the MPI standard on a 
network of workstations; it has also been implemented on a multiprocessor ma- 
chine) . 



Parallel vs Serial GA 

fitn«6S 

40 




Fig. 5. Gomparison of convergence rate of parallel DGA and single population version. 
(If computing speed up rate (nearly 9) were applied, parallel DGA performance gain 
would appear even more dramatic) 
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Direct Airfoil Optimisation Results Application of the DGA to problems 
of direct airfoil optimisation for low speed and transonic cases are described in 
[6], [11]. For both inverse and direct optimisation problems however, the use of 
an agent to supervise the operation of the DGA has been found to improve the 
performance, as described next. 

5 Agent Supervision of DGA 

The better ability of the DGA at maintaining population diversity appears to 
account for its notable gain in performance over the single population GA. Even- 
tually however, the population on each island converges. Adding an agent to su- 
pervise the operation of the parallel GA provides a capability whereby the DGA 
can adapt more generally than is possible with a sequential GA. Although adap- 
tation can be built into a GA (e.g. in the adaptive operator htness of Davis [4], 
the use of agents provides a more general framework by decoupling the tasks of 
higher level supervision from the lower level optimisation. The agent supervised 
paradigm is very well suited to a distributed computing environment, where 
agents can direct the operation of the GA on local or global populations, and 
can additionally direct processing resources. At one extreme, the agent layer may 
be combined with the DGA software to execute as a single (albeit distributed) 
entity, or at another, it may run as an entirely separate distributed program, 
communicating with the DGA by reading external output messages and writing 
to action inputs. 

A simple agent supervision of the DGA is as follows. The agent receives 
status messages from the DGA, (i.e. generation number, measures of popula- 
tion convergence, htness changes etc.), and parameter settings (local mutation 
and crossover rates/types etc.). The agent layer then instructs the DGA to take 
either global or local action. Examples of such actions could be the introduc- 
tion of a mechanism to improve diversity between island populations, actions to 
favour specihc local niches within islands, or actions to improve the parallel load 
balancing by adjusting the deme placing or sizing. 

5.1 Infection Agent 

A simple implementation of agent supervision of the DGA which was used for 
direct airfoil design employs a single agent supervisor [15] to act as a vector 
for infections, with low population diversity encouraging epidemics. On infected 
islands, individuals close to the global best have greatly reduced htness, and 
undergo increased mutation; results [15] shows the pattern of infection changes 
dynamically. When implemented for the problem of inverse design optimisa- 
tion, there was a gain in late solution convergence, beginning just beyond the 
point when the population on all the islands initially converges towards a global 
‘champion’. 

Applying the procedure to direct design (viscous L/D optimisation of a low 
speed airfoil section, at operating points of 3 and 8 degrees incidence, and a 
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Reynolds number of 4 million, with a moment constraint (|Cm| < 0.97)) pro- 
duced the shape and Cp distributions shown in hg. 6, (though this is not yet 
fully a converged solution). 





Fig. 6. L/D optimization: Distribution of Cp on the surface of the ‘best’ airfoil. This 
computation was performed using XFOIL. 



The addition of the infection in these problems was found to show a clear 
improvement over the solution obtained in [11] for corresponding effort; the 
improvement in the htness after island convergence is shown in hg. 8a further 
below. 

5.2 Evolutionary Agent Supervision 

In the learning classiher method presented earlier, adaptation occurs through 
repetitively solving a problem or class of problems. The agent supervision can 
adapt dynamically however, especially if a large number of generations are to be 
evaluated. 



Agent or 
Multi-Agent 




Fig. 7. Competing agent supervised DGA 



For example, one may place several islands in a group under the control of one 
agent, and other groups of islands under the control of other agents, with each 
agent adopting different strategies. If migration between groups is eliminated, the 
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relative improvement over a number of generations may be compared. The worse 
performing agents then modify either the parameter settings or the rules which 
they apply to the DGA under their control, and may also replace their population 
partly or fully with that of the best group. An example of the effectiveness of the 
approach is shown in hg.Sb, where the simple agent infection approach described 
previously is supervised by a pair of competing agents. A population of 200 (10 
individuals per deme; demes connected in a 4 x 5 array) was used in this case 
for the inverse design of a NACA 0012 airfoil. The agents apply different genetic 
operators to their respective groups of subpopulations, and their performance 
as managers is compared after a certain number of generations. In the example, 
only the mutation rate was altered by the supervising agents; at a lower level, 
the infection strategy was still implemented. As can be seen from hg. 8b, these 
early results are encouraging; further work will consider the effects of controlling 
different parameters, and different competition mechanisms. 




Generation Generation 



(a) 



(b) 



Fig. 8. Agent supervision: (a) Comparison of DGA and simple agent supervised DGA; 
(b)Comparison of simple agent and competing agent supervised DGA. (Note difference 
in fitness scales; results in (b) are also averaged over more trials) 



6 Conclusion 

The distributed genetic algorithm (DGA) has been applied to a number of aero- 
nautical design optimisation problems. Earlier results indicated that the method 
has better convergence behaviour than the single population GA; the present 
work outlines agent supervision strategies to improve the DGA further. A learn- 
ing classiher scheme applied to the single population GA showed some improve- 
ment, but appears very costly at present. In contrast, the use of competing 
agents which evolve appears quite promising for further investigation. Applica- 
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tions of the DGA to low speed airfoil optimisation, demonstrate that the method 

is straightforward to implement, and can be easily applied to different problems. 
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Abstract. The set covering problem is a paradigmatic NP-hard com- 
binatorial optimization problem which is used as model in relevant ap- 
plications, in particular crew scheduling in airline and mass-transit com- 
panies. This paper is concerned with the approximated solution of large 
scale set covering problems arising from crew scheduling in airline com- 
panies. We propose an adaptive heuristic-based evolutionary algorithm 
whose main ingredient is a mechanism for selecting a small core subprob- 
lem which is dynamically updated during the execution. This mechanism 
allows the algorithm to find covers of good quality in rather short time. 
Experiments conducted on real-world benchmark instances from crew 
scheduling in airline companies yield results which are competitive with 
those obtained by other commercial/academic systems, indicating the 
effectiveness of our approach for dealing with large scale set covering 
problems. 



1 Introduction 



The set covering problem (SCP) is one of the oldest and most studied NP-hard 
problems (cf. [14]). 

Given a m-row, n-column, zero-one matrix (oy), and an n-dimensional inte- 
ger vector (wj), the problem consists of finding a subset of columns covering all 
the rows and having minimum total weight. A row i is covered by a column j 
if the entry is equal to 1. This problem can be formulated as a constrained 
optimization problem as follows: 



• • • 

minimize '^j^j 

subject to the constraints 



e {0,1} j = l,...,n, 
> 1 * = 1, ■ ■ ■ ,TO- 
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The variable Xj indicates whether column j belongs to the solution {xj = 1) 
or not {xj = 0). The m constraint inequalities are used to express the require- 
ment that each row be covered by at least one column. The weight Wj is a 
positive integer that specifies the cost of column j. When all Wj’s are equal to 
1, then the SCP is called unicost SCP. 

Relevant practical applications of the SCP include crew scheduling [1,2, 12, 
15]: find a set of pairings having minimum-cost which covers a given set of trips, 
where a pairing is a sequence of trips that can be performed by a single crew. 
A widely used approach to crew scheduling works as follows. First, a very large 
number of pairings is generated. Next, a SCP is solved, having as rows the trips 
to be covered, and as columns the pairings generated. When this approach is used 
in mass-transit applications, very large scale SCP instances may arise, involving 
thousands of rows and millions of columns. 

The most successful heuristic algorithms for large scale SCP’s are based on 
Lagrangian relaxation [13]. Lagrangian relaxation is used to compute the score of 
a column according to its likelihood to be selected in an optimal solution. These 
scores are employed in simple greedy heuristics for computing a solution. A very 
effective heuristic algorithm for large scale SCPs based on this approach is [7]. 
We refer the reader to [8] for a recent survey on exact and heuristic algorithms 
for SCP. All effective heuristics for large scale SCP’s act on a subset of the 
columns, called core, which is selected before the execution of the algorithm. In 
the static approach the core remains the same during the execution (cf. [6, 11]), 
while in the dynamic approach it is updated using an adaptive mechanism (e.g. 
[7,10,9]). 

In this paper we propose a novel heuristic algorithm for large scale SCPs 
arising from crew sheduling problems in airline companies. At each iteration a 
near optimal cover is constructed using the information provided by the previous 
iterations to guide the search. The final solution is the best cover found in all 
the iterations. 

Given a problem instance, the algorithm extracts an initial core from the 
set of columns given in the input. Then the algorithm consits of the iterated 
application of the following three steps: 1) First, an approximated solution to 
the actual SCP core is constructed by means of a novel greedy heuristic. 2) 
Next, a local search optimization algorithm is applied to the resulting solution. 
3) Finally, some columns that occur in the best solution found in all iterations up 
to now are selected for forming the initial partial solution for the next iteration. 

The size of the core is determined by an adaptive size parameter, while 
the selection of a column is specified by a suitable merit criterion. During the 
execution, the score of the columns is modified as well as the size parameter, 
and the core is dynamically updated. 

This algorithm can be viewed as a hybrid (1 + 1) steady-state evolutionary 
algorithm, where at each iteration a child is generated from the parent using 
the above described heuristic, and the best between the parent and the child 
survives. 
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In order to assess the performance of the algorithm, we conduct extensive 
experiments on real-world problem instances arising from crew scheduling in air- 
lines, as well as on other benchmark instances from the literature. The results 
of the experiments are rather satisfactory: our algorithm is able to find covers 
of very good quality in a short amount of time, yielding results which are com- 
petitive with those reported by the best industrial as well as academic methods 
for solving large set covering problems. 

The rest of the paper is organized as follows. In the next subsections we 
briefly discuss some related work, and set up the notation and terminology used 
throughout the paper. In Section 2 we introduce the overall method and present 
in detail the four main modules of the algorithm. In Section 3 we report the 
results of extensive computational experiments. We conclude with some final 
remarks on the present investigation and on future work. 



1.1 Related Work 

An experimental comparison of the most effective exact and heuristic algorithms 
for the weighted SCP is given in a recent paper by Caprara et al [8]. 

A rather effective heuristic algorithm based on Lagrangian relaxation is the 
CFT algorithm [7] by Caprara et al. This algorithm has been tested also on 
large scale problem instances arising from crew scheduling in railway, yielding 
rather satisfactory results. In [15] a approximation algorithm for solving large 
0-1 integer programming problems is proposed. This algorithm is used in the 
CARMEN system for airline crew scheduling, a industrial system used by several 
major airlines. 

Research based on evolutionary computation includes the following two pa- 
pers. 

Beasley and Chu in [6] introduce a genetic algorithm for the SCP. The authors 
employ a representation where a chromosome is a bit string of lenght equal to 
the number of columns, one bit for each column, representing the set of columns 
whose bit in the string are equal to 1 . The algorithm employs a heuristic repair 
mechanism for transforming infeasible chromosomes into solutions. Moreover, a 
core is used for constructing the chromosomes of the initial population. 

A genetic algorithm based on a non-binary representation has been proposed 
by Eremeev in [11]. Here a chromosome is a string of lenght equal to the number 
of rows, where the Tth entry contains (the index of) a column covering the i- 
th row. As a consequence, all chromosomes are feasible solutions, thus they do 
not need to be repaired as in [6]. Moreover, heuristics are used for eliminating 
redundant columns as well as for defining the crossover operator. 

In Section 3 we will compare experimentally the above mentioned algorithms 
with the algorithm introduced in this paper. 



1.2 Notation and Terminology 

In order to describe our method, we use the following terminology and notation. 
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In the sequel, the indexes i,j denote a generic row and column, respectively. A 
column will also be denoted by c, and a row by r, possibly subscripted. Moreover, 
S denotes a set of columns. 

Let cov{S) be the set of rows that are covered by the columns in S: 
cov(S) = {i I Gij = 1 for some j G S' }. 

For simplicity, we write cov{j) instead of cov{{j}). We say that a column j 
is redundant with respect to S if cov(S \ {j}) = cov(S). 

A partial cover (also called partial solution) is a set of columns containing 
no redundant column. 

Let cov(j,S) be the set of rows which are covered by column j, but are not 
covered by any column in S \ {j}: 



cov(j, S) = I Oij = 1 and Oiji = 0 for all j' G S \ {j} }. 

Moreover, let minjweight{i) be the minimum weight of the columns that 
cover i: 

minjweight{i) = minimum {wj \ i G cov(j) }. 

We can now define the function cov.val, called cover value, which is used to 
evaluate a column j with respect to a partial cover S in order to select a column 
to be added (resp. removed) to (resp. from) S: 



cov-val(j, S) = min.weight{i). 

zG cov{j,S) 

A convenient property of cov-val is that cov-val(j, S) = cov.val(j,S \ {j}). 
This allows one to compute the cover value of a column without taking into ac- 
count whether it belongs to the partial solution S or not. Moreover, we can char- 
acterize the redundancy of a column by means of the condition cov-val(j, S) = 0. 

The cover value is used to define the selection value sel-val{j, S) of a column 
j with respect to the partial cover S: 



sel-va 




Lim if i redundant wrt 

Wj / cov-val(j, S) otherwise. 



S, 



The selection value of redundant columns is set to a very big constant Lim. 
In this way, redundant columns do not have any chance of being selected. 



2 The Overall Method 

The algorithm we propose consists of an iterated procedure, where each iteration 
generates an approximated solution using only columns from the actual core. 

Roughly, at each iteration a greedy heuristic is used to construct incremen- 
tally a cover starting from a partial cover: in the first iteration the partial cover 
is empty, while in the following iterations the partial cover is a proper subset 
of the best cover found in all iterations up to now. The cover found after the 
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application of the greedy heuristic is given as input to an optimization procedure 
which tries to improve the partial solution. The core is updated from time to time 
during the execution. The final result is the best cover found in all iterations. 

The corresponding algorithm WSCP (Weighted Set Covering Problem) is 
illustrated below in pseudo-code, where Sbest represents the best cover found 
so far, S denotes the actual partial cover, and value (S) is the sum of the weights 
of the columns of S, that is, Therefore the optimal cover is the cover 

S having minimum value (S). 

FUNCTION WSCPO 
BEGIN 

REC0MPUTE_C0RE() ; 

Sbest <- { l..ncol }; 

S <- { }; 

FOR 1 . . param.number_of_ iterations DO 

IF ( core_selection() ) REC0MPUTE_C0RE() ; ENDIF; 

S <- GREEDY (S); 

S <- OPTIMIZE(S); 

IF ( value (S) <= value (Sbest) ) THEN Sbest <- S; ENDIF; 

S <- SELECT_PARTIAL_COVER(Sbest) ; 

ENDFOR 
RETURN Sbest 

END 



2.1 Greedy Heuristic 

Our greedy heuristic GREEDY is described in pseudo-code below. Lines starting 
with ”//” are comments. The algorithm constructs a solution (a cover), starting 
from a (possibly empty) partial cover S. Columns are added (resp. removed) to 
(resp. from) S until S covers all the rows. 

// extend S until it is a cover: 

FUNCTION GREEDY ( var S ) 

BEGIN 

WHILE ( S is not a cover ) DO 

// select and add one column to S 
S <- S + select_add() ; 

// remove 0 or more columns from S 
WHILE ( remove_is_okay 0 ) DO 
S <- S - select_rmv() ; 

END WHILE 
END WHILE 

// S is a cover, without redundant columns 
return S ; 



END 
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The function select jadd selects a column j not in S having minimum selection 
value sel-val(j, S). 

The test removeJs-okay determines whether columns should be removed 
from S. If S is empty it returns false; if S contains at least one redundant 
column then it returns true; otherwise, with probability param.pjrmv (typical 
value 0.3) it returns true, otherwise false. 

Finally, the function select jrmv selects a column in S having maximum se- 
lection value. 

2.2 Local Optimization 

The local optimization procedure OPTIMIZE is based on the following idea. 
Given a cover S, suppose there is a column j ^ S such that S' U {j} contains 
at least two columns other than j, say j\,. . . ,ji, with I > 2 that are redundant, 
and such that the sum of their weights is greater than the weight of j, that is, 
i '^jk > ■ Then (S \ jji , . . . , ji}) U {j} is a better cover than S. In this 

case we call j a superior column. The gain of j is defined by 

gain{j)= ^ Wj^-Wj. 

k^l,...,l 

So a best superior column is the one having highest gain. Note that the opti- 
mization procedure operates on a cover containing no redundant columns. The 
optimization algorithm OPTIMIZE in pseudo-code is given below. 

// S is a cover, without redundant columns 
FUNCTION OPTIMIZE! var S ) 

BEGIN 

Sup <- select_superior 0 ; 

WHILE ( Sup not empty) DO 

// select best column from Sup 
best <- select_best () ; 

Sup <- Sup - best; 

// add superior and remove redundant columns from S 
IF ( best superior ) 

S <- S + best; 

S <- S - select_redundant 0 ; 

END IF 
END WHILE 

// S is a cover, without redundant columns 
return S ; 

END 



Eirst, the function selectsuperiorQ is used, which generates the list Sup 
consisting of all the superior columns ordered in decreasing order according to 
their gain. Next, the list Sup is scanned in the WHILE loop. At each iter- 
ation, the head of Sup is removed and memorized in the variable best using 
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the function select .best {) . If the selected column is still superior (that is if the 
test best superior is satisfied) then it is added to S, and the set of redun- 
dant columns are removed from the resulting partial cover S using the function 
select. redundant{) . 

2.3 Restoring Part of the Actual Best Solution 

In the first iteration of WSCP the heuristic GREEDY constructs a cover start- 
ing from the empty set; in the following iterations, GREEDY builds a cover 
starting from a subset of the best cover found so far. Eor a column j, we keep 
track of the number chosen{j) of times that j has been part of a best solu- 
tion. The function SELECT_PARTIAL_COVER considers the set E of so-called 
elite columns, consisting of those columns j of the best solution Sbest such that 
cov.val{j,Sbest) > Wj. Then SELECT_PARTIAL_COVER selects from E the 
set of columns having low chosen(j) (in our implementation chosen(j) has to be 
smaller than J2j chosen{j) / {neli * 10), where neli is the number of elements of 
E) while the remaining columns of E are selected with a probability that is set 
to a random value between 0.1 and 0.9. 

2.4 Selecting the SCP Core 

This is a fundamental step in the design of an algorithm for dealing with large 
SCP instances. We introduce the following method for constructing an SCP core, 
which has been implemented in the function RECOMPUTE_CORE. 

The SCP cover is constructed from the empty set by incrementally adding 
columns according to the following criterion. Columns are selected in increasing 
order according to their selection value. Suppose column j has been selected: 

1. if j is an elite column then with probability close to 1 it is added to the 
actual SCP core; 

2. otherwise, j is added if there exists a row i such that j covers i and Wj < 
min.weight{i) • Kq, with Kq a given constant real value greater or equal 
than 1; 

3. otherwise, j is added if there exists a row i such that j covers i and i is 
covered by less than K\ columns of the actual SCP core, with K\ a given 
constant integer value greater or equal than 1. 

Note that Kq,K\ are parameters which are chosen depending on the class 
of problems one considers. Condition 3 implies that the SCP core contains for 
each row, at least the first K\ best columns (according to the ordering induced 
by the selection value function) that cover that row. 

The function core.selection{) determines when the actual SCP core has to 
be recomputed. In our implementation, we recompute the SCP core every 100 
iterations of WSCP. 

During the execution of GREEDY, when ninety per cent of a cover has been 
constructed, the min.weight of those rows that are not yet covered is increased 
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by a small quantity (in our implementation min.weight{i) is multiplied by 1.1). 
This affects the selection value of the columns, hence their order of selection in 
the construction of the SCP core changes during the execution of the overall 
algorithm WSCP. 

3 Experimental Evaluation 

The algorithm WSCP has been tested on large set covering problems arising 
from crew scheduling applications in various airline companies. Moreover, we 
have considered the weighted SCP instances from the OR library maintained by 
J.E. Beasley ^ . These instances are considered standard benchmarks for testing 
the effectiveness of exact and heuristic algorithms for the SCP. In particular, they 
have been used in [8] for comparing experimentally various exact and heuristic 
algorithms for SCP. 

WSCP has been implemented in C++. The algorithm was run on a Sun 
Ultra 10 (UltraSPARC-IIi 300MHz). 

The results of the experiments are based on 10 runs on each problem instance 
of the OR Library, and on 5 runs on the other instances. In each table, the 
entry labeled Id contains the name of the problem instance. The label BK 
denotes the best known solution for that instance; Bst denotes the best result 
found by the algorithm; Fbst indicates the frequency of obtaining the best 
solution in the performed runs; Apd denotes the average percentage deviation 
"^k^iia{zk — z*)l{\Q ■ z*) ■ 100%, where Zk is the solution found in the A;-th run, 
and 2 :* is the optimal or best known solution. Tbst denotes the average cpu 
time for obtaining the best solution Bst, while Tsoi denotes the average cpu 
time for finding a solution. Finally, Ibst and Isol denote the average number of 
iterations of obtaining the best solution Bst, and a solution, respectively. 



3.1 Experiments on Airline Crew Scheduling Problem Instances 

We consider three sets of benchmark instances from real-world airline crew 
scheduling problems. A set of instances from a major airline company, here 
called AIR instances, the airline scheduling instances from Wedelin [15], and the 
instances from Balas and Carrera [3]. The characteristics of these problems are 
reported in Tables 2, 3, and 7, respectively. Observe that in many instances, like, 
e.g., the Wedelin instances, the weights of the columns are very large numbers, 
because the weight represents the cost of a pairing and takes into account several 
factors. 

We compare experimentally WSCP with the industrial system used by an 
airline company on the AIR instances, with the CFT algorithm by Caprara et al 
[7], and with the Wedelin algorithm [15]. The results of the experiments are given 
in Tables 2, 4, 5, and 7. Note that the results for the Wedelin and CFT algorithms 
are taken from the paper [7], where the cpu time is estimated in DECstation 



^ see http://mscmga.ms.ic.ac.uk/jeb/orlib/scpinfo.html 
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5000/240 CPU seconds. Only the value of the best solution is reported. For the 
CFT algorithm, the time for finding the best solution is given, while for the 
Wedelin algorithm, only the overall execution time Texe of the algorithm is 
reported. The authors do not specify the setting of the various parameters in 
their algorithms, and the total number of trials performed. 



Id 


Rows 


Columns 


Density (%) 


Weight Range 


AOI 


5265 


258303 


0.167 


1319-35302 


A02 


3878 


I944I 


0.135 


1437-37206 


A03 


4965 


40580 


0.092 


1337-37148 


A04 


4916 


79481 


0.123 


1460-37142 


A05 


4656 


72377 


0.126 


I4II-3725I 


A06 


1971 


23741 


0.135 


1437-37037 


A07 


4203 


32363 


0.15 


1319-36370 


AOS 


4320 


45286 


0.18 


1345-36370 


A09 


4287 


50047 


0.19 


1361-36370 


AIO 


4369 


49525 


0.18 


1344-36370 


All 


150 


389388 


5.55 


1800-18768 


AI2 


682 


642613 


1.45 


1630-19000 



Table 1. Characteristics of AIR instances 



Id 


Industry 

Bst 


Bst 


Fbst 


Vi 

Apd 


ASCP 

Tbst 


Tsoi 


Ibst 


Isol 


AOI 


I635I667 


I635I667 


1.0 


0.0 


550.9 


550.9 


919.8 


919.8 


A02 


12879297 


12879297 


1.0 


0.0 


131.0 


131.0 


822.4 


822.4 


A03 


15663720 


15663688 


1.0 


0.0 


254.2 


254.2 


1004.4 


1004.4 


A04 


I6II0608 


I6II0608 


1.0 


0.0 


363.1 


363.1 


1085.0 


1085.0 


A05 


I63I524I 


16315070 


0.3 


0.0001 


923.5 


848.1 


3501.6 


3203.2 


A06 


I3I625II 


13162511 


1.0 


0.0 


156.9 


156.9 


907.6 


907.6 


A07 


I330I520 


13301520 


1.0 


0.0 


200.9 


200.9 


945.6 


945.6 


AOS 


I35I0606 


13510584 


1.0 


0.0 


254.4 


254.4 


946.2 


946.2 


A09 


13489489 


13489489 


1.0 


0.0 


235.4 


235.4 


944.2 


944.2 


AIO 


I357I530 


13571530 


1.0 


0.0 


237.3 


237.3 


933.8 


933.8 


All 


247775 


247775 


1.0 


0.0 


224.2 


224.2 


1087.8 


1087.8 


AI2 


732587 


732587 


0.3 


0.11 


1064.9 


896.8 


1460.6 


1167.4 



Table 2. Results for AIR instances 



On three AIR instances WSCP found a solution which is better than the 
best solution found by the industrial system, while on the other instances WSCP 
found solutions of equal value as those found by the industrial system. 
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Id 


Rows 


Columns 


Density (%) 


Weight Range 


B727scratch 


29 


157 


8.2 


1600-11850 


ALITALIA 


118 


1165 


3.1 


2200-2110900 


A320 


199 


6931 


2.3 


1600-2111450 


A320COC 


235 


18753 


1.9 


1900-1812000 


SASjump 


742 


10.370 


0.6 


4720-55849 


SASD9imp2 


1366 


25032 


0.3 


3860-35200 



Table 3. Characteristics of Wedelin instances 



Id 


BK 


CF" 

Bst 


r 

Tbst 


Wede 

Bst 


lin 

Texe 


B727scratch 


94400 


94.400 


0.3 


94400 


4.7 


ALITALIA 


27258300 


27258300 


6.2 


27258300 


37.2 


A320 


1262100 


1262100 


79.5 


1262100 


216.9 


A320COC 


14495500 


14495600 


577.8 


14495500 


1023.7 


SASjump 


7338844 


7339537 


396.3 


7340777 


806.8 


SASD9imp2 


5262190 


5263640 


2082.1 


5262190 


1579.7 



Table 4. Results of CFT and Wedelin on Wedelin instances 



Id 


BK 


Bst 


Fbst 


Apd 


WSCP 

Tbst 


Tsoi 


Ibst 


Isol 


B727scratch 


94400 


94400 


1.0 


0.0 


0.018 


0.018 


38.4 


38.4 


ALITALIA 


27258300 


27258300 


1.0 


0.0 


0.63 


0.63 


106.8 


106.8 


A320 


1262100 


1262100 


IQ 


0.0 


17.34 


17.34 


326.2 


326.2 


A320COC 


14495500 


14495500 


0.2 


0.0006 


651.08 


446.20 


3494.5 


2402.0 


SASjump 


7338844 


7339541 


m 


0.02 


269.3 


200.98 


4635.0 


3454.6 


SASD9imp2 


5262190 


5263590 


m 


0.04 


741.9 


608.452 


4603.0 


3671.4 



Table 5. Results of WSCP on Wedelin instances 
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Id 


Rows 


Columns 


Density (%) 


Weight Range 


AA03 


106 


8661 


4.05 


91-3619 


AA04 


106 


8002 


4.05 


91-3619 


AA05 


105 


7435 


4.05 


91-3619 


AA06 


105 


6951 


4.11 


91-3619 


AAll 


271 


4413 


2.53 


35-2966 


AA12 


272 


4208 


2.52 


35-2966 


AA13 


265 


4025 


2.60 


35-2966 


AA14 


266 


3868 


2.50 


35-2966 


AA15 


267 


3701 


2.58 


35-2966 


AA16 


265 


3558 


2.63 


35-2966 


AA17 


264 


3425 


2.61 


35-2966 


AA18 


271 


3314 


2.55 


35-2966 


AA19 


263 


3202 


2.63 


35-2966 


AA20 


269 


3095 


2.58 


35-2966 


BUSl 


454 


2241 


1.89 


120-877 


BUS2 


681 


9524 


0.51 


120-576 



Table 6. Characteristics of Balas and Carrera instances 



Id 


Cl 

Bst 


’T 

Tbst 


Bst 


Fbst 


Apd 


wsc 

Tbst 


P 

Tsoi 


Ibst 


Isol 


AA03 


33155 


61.0 


33155 


1.0 


0.0 


1.26 


1.266 


40.0 


40.0 


AA04 


34573 


3.6 


34573 


1.0 


0.0 


1.73 


1.73 


74.6 


74.6 


AA05 


31623 


3.1 


31623 


1.0 


0.0 


0.48 


0.48 


9.6 


9.6 


AA06 


37464 


5.2 


37464 


1.0 


0.0 


2.67 


2.67 


128.2 


128.2 


AAll 


35384 


193.7 


35384 


1.0 


0.0 


19.11 


19.11 


755.4 


755.4 


AA12 


30809 


53.8 


30809 


1.0 


0.0 


7.88 


7.88 


350.8 


350.8 


AA13 


33211 


8.3 


33211 


1.0 


0.0 


2.32 


2.32 


103.8 


103.8 


AA14 


33219 


30.3 


33219 


1.0 


0.0 


11.74 


11.74 


557.8 


557.8 


AA15 


34409 


18.8 


34409 


1.0 


0.0 


8.92 


8.92 


485.6 


485.6 


AA16 


32752 


33.6 


32752 


1.0 


0.0 


4.63 


4.63 


257.4 


257.4 


AA17 


31612 


10.9 


31612 


1.0 


0.0 


4.69 


4.69 


262.2 


262.2 


AA18 


36782 


13.5 


36782 


0.1 


0.01 


17.1 


6.94 


1108.0 


433.0 


AA19 


32317 


5.9 


32317 


1.0 


0.0 


2.73 


2.73 


175.4 


175.4 


AA20 


34912 


13.6 


34912 


1.0 


0.0 


4.76 


4.76 


318.4 


318.4 


BUSl 


27947 


5.0 


27947 


1.0 


0.0 


8.19 


8.19 


382.6 


382.6 


BUS2 


67760 


19.2 


67760 


1.0 


0.0 


37.24 


37.24 


616.2 


616.2 



Table 7. Results of CFT and WSCP on Balas and Carrera instances 
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On the instances from Wedelin the performance of WSCP is comparable to 
the one of the CFT and Wedelin algorithms. 

Finally, on the instances from Balas and Carrera, both WSCP and CFT are 
always able to find the optimal solution. In the AA instances WSCP is faster 
that CFT, while in the BUS instances CFT finds the optimum in a shorter time. 

The results of the experiments indicate that WSCP is a rather powerful tool 
for solving large real-life airline crew scheduling problems. 

3.2 Experiments on the OR Library SCP Instances 

We consider the families A-D from [4] , and the NRE-NRH from [5] , consisting of 
randomly generated SCP instances. Each class contains 5 instances. The values 
of the characteristic parameters of these problem classes, like number of rows 
and columns, are given in Table 8. 

We compare experimentally WSCP with the genetic algorithms by Beasley 
and Chu [6], and by Eremeev [11], and with the CET algorithm by Caprara et 
al [7]. The results of the experiments are summarized in Tables 9, 10, and 11. 

The results for the CET, Beasley Chu, and Eremeev algorithms are from [11]. 
In particular, the cpu time is estimated in lOOMHz Pentium CPU seconds. 

All the algorithms are able to solve the instances of the classes A-D. On these 
instances, WSCP seems to have a more robust behaviour that the two genetic 
algorithms, finding the optimum in each of the 10 trials. The performance of 
WSCP on the other problem instances of classes E-H is rather satisfactory, both 
in terms of quality of the solutions as well as running time. On each instance, 
WSCP is able to find the optimum or best known solution, while the two genetic 
algorithms BC and Er do not find the optimum value on instances HI and H2. 
Moreover, WSCP finds the solutions for instances in the harder classes G and H 
in a much shorter time than all the other algorithms. 



Id 


A 


B 


C 


D 


E 


F 


G 


H 


Rows 


300 


300 


400 


400 


500 


500 


1000 


1000 


Columns 


3000 


3000 


4000 


4000 


5000 


5000 


10000 


10000 


Density (%) 


2 


5 


2 


5 






2 


5 


Weight Range 


1-100 


1-100 


1-100 


1-100 


1-100 


1-100 


1-100 


1-100 



Table 8. Characteristics of Classes A, B, C, D 



4 Conclusion 

In this paper we have introduced a novel heuristic method for solving large 
weighted set covering problems. The results of the experiments indicate that 
WSCP is able to find covers of satisfactory quality in short running time. 
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Id 


CFT 

Tbst 


Be 

Fbst 


asley 

Apd 


Chu 

Tsoi 


Fbst 


Ere 

Apd 


meev 

Tbst 


Tsoi 


Fbst 


Apd 


ws 

Tbst 


5CP 

Tsoi 


Ibst 


Isol 


!□ 


47.15 


0^ 




65.98 


QQI 




EHtm 


71.8 












Ujggjjl 


!□ 


3.34 






68.63 


















7.8 


7.8 


IQ 


29.23 




QEQ 


87.93 






53.50 


52.40 










72.6 


72.6 


IQ 


7.64 


0^ 




101.70 


















26.0 


26.0 



Table 9. Results for Classes A, B, C, D 



Id 


BK 


CFT 


Beasley Chu 




Eremeev 








Bst 


Tbst 


Bst 


Fbst 


Apd 


Tsoi 


Bst 


Fbst 


Apd 


Tbst 


Tsoi 


El 


29 


29 


11.5 


29 


1.0 


0.0 


16.9 


29 


1.0 


0.0 


1.0 


1.0 


E2 


30 


30 


180.5 


30 


0.4 


2.0 


266.9 


30 


1.0 


0.0 


94.8 


94.8 


E3 


27 


27 


41.7 


27 


0.3 


2.6 


85.1 


27 


1.0 


0.0 


23.1 


23.1 


E4 


28 


28 


11.6 


28 


1.0 


0.0 


238.5 


28 


1.0 


0.0 


11.0 


11.0 


E5 


28 


28 


16.2 


28 


1.0 


0.0 


15.5 


28 


1.0 


0.0 


2.1 


2.1 


FI 


14 


14 


14.7 


14 


1.0 


0.0 


33.8 


14 


1.0 


0.0 


7.9 


7.9 


F2 


15 


15 


13.8 


15 


1.0 


0.0 


34.5 


15 


1.0 


0.0 


1.3 


1.3 


F3 


14 


14 


110.0 


14 


1.0 


0.0 


117.9 


14 


1.0 


0.0 


55.4 


55.4 


F4 


14 


14 


13.7 


14 


1.0 


0.0 


92.6 


14 


1.0 


0.0 


20.4 


20.4 


F5 


13 


13 


89.0 


13 


0.3 


5.4 


67.1 


13 


0.3 


5.4 


497.4 


151.2 


G1 


176 


176 


65.0 


176 


0.2 


1.0 


451.3 


176 


0.7 


0.3 


115.0 


96.0 


G2 


154 


154 


346.6 


155 


0.5 


1.5 


159.3 


154 


0.5 


0.65 


318.3 


226.6 


G3 


166 


166 


432.7 


166 


0.1 


1.1 


312.1 


166 


0.1 


0.8 


627.6 


319.1 


G4 


168 


168 


105.0 


168 


0.2 


1.4 


665.4 


168 


0.4 


0.7 


160.0 


172.5 


G5 


168 


168 


105.0 


168 


0.2 


0.8 


242.6 


168 


0.7 


0.05 


161.2 


170.4 


HI 


63 


63 


642.1 


64 


1.0 


1.6 


743.0 


64 


1.0 


1.6 


90.5 


90.5 


H2 


63 


63 


392.5 


64 


1.0 


1.6 


234.3 


64 


1.0 


1.6 


34.7 


34.7 


H3 


59 


59 


690.4 


59 


0.9 


0.2 


796.6 


59 


1.0 


0.0 


493.2 


493.2 


H4 


58 


58 


105.1 


58 


0.4 


91.6 


62.9 


58 


1.0 


0.0 


218.2 


218.2 


H5 


55 


55 


68.8 


55 


0.9 


0.2 


198.6 


55 


1.0 


0.0 


25.2 


25.2 



Table 10. Results of CFT, Beasley and Chu, and Eremeev on Classes E, F, G, FI 
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Id 


m 


Bst 


Fbst 


Apd 


WS 

Tbst 


CP 

Tsoi 


Ibst 


Isol 


El 


29 


29 


1.0 


0.0 


1.8 


1.8 


2 


2 


E2 


30 


30 


1.0 


0.0 


2.7 


2.7 


62.9 


62.9 


E3 


27 


27 


1.0 


0.0 


2.3 


2.3 


48.3 


48.3 


E4 


28 


28 


1.0 


0.0 


2.1 


2.1 


31.1 


31.1 


E5 


28 


28 


1.0 


0.0 


1.8 


1.8 


5.0 


5.0 


FI 


14 


14 


1.0 


0.0 


3.6 


3.6 


19.6 


19.6 


F2 


15 


15 


1.0 


0.0 


3.6 


3.6 


9.0 


9.0 


F3 


14 


14 


1.0 


0.0 


6.2 


6.2 


153.2 


153.2 


F4 


14 


14 


1.0 


0.0 


3.6 


3.6 


13.1 


13.1 


F5 


13 


13 


1.0 


0.0 


34.1 


34.1 


2061.1 


2061.5 


G1 


176 


176 


1.0 


0.0 


2.2 


2.2 


29.7 


29.7 


G2 


154 


154 


0.5 


1.1 


8.9 


4.0 


315 


107.9 


G3 


166 


166 


0.2 


0.6 


30.2 


14.9 


1433.5 


640.3 


G4 


168 


168 


0.4 


0.8 


18.4 


25.0 


812.3 


1114.7 


G5 


168 


168 


0.9 


0.6 


5.9 


5.7 


207.7 


197.4 


HI 


63 


63 


0.2 


1.1 


9.5 


11.7 


161.5 


269.2 


H2 


63 


63 


1.0 


0.0 


50.4 


50.4 


1872 


1872 


H3 


59 


59 


0.3 


1.1 


25.2 


21.6 


778.3 


678.9 


H4 


58 


58 


0.8 


0.3 


28.1 


23.9 


1016.3 


834.1 


H5 


55 


55 


1.0 


0.0 


5.5 


5.5 


64.1 


64.1 



Table 11. Results of WSCP on Classes E, F, G, H 



In all the experiments we have worked with a core which is a proper subset 
of the set of all columns. The size of the core depends on the problem instance. 
However, in general a small fraction (which varies from 10 per cent to 50 per 
cent) of the set of columns is used as core. Using small covers helps the efficiency 
of the algorithm. Moreover, extensive experiments with different core sizes have 
revealed a somehow counter intuitive phenomenon: in many instances, the qual- 
ity of the results become worse by using a larger core, even if the same number 
of iterations is used. This seems to indicate that the merit criterion used in 
WSCP is not the best possible, because it can make the wrong decision when 
all the columns are present in the core. We are actually investigating the use of 
alternative merit criteria and their relationship with the selection of the core. 

Future work concerns the investigation of how to tune automatically the pa- 
rameters Kq, K\ for determining the core problem, and how the value of thesecan 
be adaptively change during the execution. 
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Abstract. Optimal positioning of aircraft at a specific airport is a very 
difficult problem involving the modeling of many constraints. Lufthansa 
AG formulated this problem field for the airport Frankfurt/Main. In this 
contribution we describe the development of a tool for finding solutions to 
positioning problems automatically. Our approach consists of two parts. 
A generic airport model is developed where the notion of logical fiber- 
ings plays a basic role. The optimization task is treated by application 
of modified and extended genetic algorithms. A system has been imple- 
mented which is capable of computing concrete positioning plans that 
can be used by a human operator for further processing. This leads to a 
considerable speed up in the generation of positioning plans for aircraft 
in comparison with the former method. The application of the aircraft 
positioning tool to a real world scenario (airport Frankfurt) is briefly 
presented. 



Keywords: optimal aircraft positioning, logical fiberings, genetic algorithms, 
hybrid problem solving 

1 Introduction 

This contribution deals with the general problem of optimal positioning of air- 
craft at an airport. The problem formulation has been provided by Lufthansa 
AG for the concrete case of airport Frankfurt/Main. The main task in the field 
of aircraft positioning is to find an optimal schedule for all incoming and outgo- 
ing aircraft with respect to their position at corresponding gates of an airport. 
One has to take into account many constraints and requests, such as neigh- 
borhood relationships, runway crossings, critical passenger connections, aircraft 
types, airline requests, special gates, security constraints, and others, depending 
on particular situations. The basic task is to fulfill all these constraints and re- 
quests in an optimal way so that the yield converges to a predefined maximum. 
Our work consists of two main parts. The development of a “generic airport 
model” not depending on the choice of a specific airport - and thus reusable 
- and the application of extended and modified genetic algorithms to work on 
the optimization problem. A rather complicated fitness function (cost function) 
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has been developed in cooperation with Lufthansa. In establishing the airport 
model, the concept of logical fiberings plays a basic role. This is a logical mod- 
eling approach which allows to describe what we call the logical state space 
of a virtual airport (distribution of logics). Thus, for example, a gate and a 
position has its own “local” logic, respectively, and they can communicate via 
non-classical logical operations called transjunctions. In principle, it would be 
possible to apply methods from multi-agent systems (MAS) . In our case a gate or 
an airplane position could be interpreted as an agent, respectively. Application of 
MAS techniques is intended future work. In terms of logical fiberings an “agent” 
corresponds to a “local fiber” . For the treatment of the optimization problem we 
develop and apply modified and extended genetic algorithms. A suitable fitness 
function has been devised which encodes the basic information and constraints 
underlying the positioning task. A prototype of an aircraft positioning tool has 
been implemented. A detailed description of the generic airport model, the evo- 
lutionary optimization approach, the system design and real world applications 
can be found in [FW99]. 

As far as we know no such approach to treat the problem field as described 
above has been tried before. This has been confirmed by somebody from the 
EvoNet project. And no such implemented system existed before, as Lufthansa 
states. There is a considerable increase of performance by our system in com- 
parison with the procedure in use so far. Our PC-based system needs about one 
hour to calculate a positioning plan which can then be used by a human operator 
to process it further. In contrast to that, with the former method, the human 
planner needed about one week to establish a similar plan “by hand” supported 
by a graphical computing system, interactively. 

2 On logical fiberings 

The purpose of this section is twofold: to give a very brief introduction to the 
elementary notions of logical fiberings for later use and to bring them to the 
attention of the reader. The concept of logical fiberings originates in an indus- 
trial project - a case study on so-called polycontextural logics and their possible 
applications to complex communication and information systems. Subsequently 
we present only the elementary notion of a logical fibering and point to prac- 
tical applications in the area of cooperating agents (in particular cooperating 
robots) scenarios. A detailed introduction to logical fiberings with background 
information and motivating comments can be found in [Pfa91]. 

The notion of a logical fibering is inspired by the mathematical modeling 
language of fiber bundles. This very expressive and powerful notion integrates 
different structures, namely geometric, topological, and algebraic structures. A 
fiber bundle consists of a whole bunch of fibers which form a so-called total space 
ditributed over a base manifold. Each fiber is mapped via a projection map onto 
its base point in the base space. Thus, for example, in a vector bundle the typical 
fiber is a vector space of a given dimension. So we can say that in a fiber bundle 
vertically one does algebra and horizontally geometry, topology. 
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Concerning a logical fibering, now the idea is to use a fiber bundle and take 
logics as fibers, thus, vertically one does logic. In [Pfa91] we take as typical fiber 
a classical two valued logic. Thus a logical fibering is an abstract fiber space 
(or bundle) with typical fiber F = L, a classical first order logical space or a 
(sub-)space generated by a set of formulas. The base space B of the fibering will 
often be denoted hy B = I, the indexing set, the total space is denoted by E, 
and 7 T : E — > I is the corresponding projection map. For i £ I we have the fiber 
7T“^(i) over i, namely = {x £ E\tt{x) = i}. It has the structure of a logic 

as mentioned above. We note that E is decomposed into the fibers for all 

i £ L 

The simplest form of a fibering is the ’’trivial fibering” having total space 
E = I X L, base space I and Tr{i,l) = i. Therefore the fiber over i £ I is 
Li := Tr~^{i) = {i} x L. Such a trivial fibering is a “parallel system” of logics L, 
over the index set I as base space. We can think of reasoning processes running 
in parallel within each fiber L, = A fiber L, is interpreted as a local 

logical system (a subsystem of the whole fibering) . Transition (communication) 
betweeen fibers is described with the help of suitable maps (cf. [Pfa91]). Such a 
trivial fibering as previously considered will also be called ’’free parallel system” 
with total space the disjoint union of the fibers L, . Each subsystem L, has local 
classical truth values 12, = {Tj, Fi}. The global set of truth values is denoted by 
In a free parallel system (2^ is just the disjoint union of the Qi for i £ I. 

Logical connectives can be introduced by taking ’’fiberwise” logical oper- 
ations. For example, one can form logical expressions like the following for a 
system with three fibers (using vector notation): {x\ Ayi,X 2 => 1 / 2 , 2:3 V {^ys)), 
etc.. For more details we refer to [Pfa91]. 

A special nonclassical bivariate operation arises naturally: a local pair (xi,yi) 
in Li X Li, i £ I, can be mapped into different subsystems La,Lp,.... Taking 
truth values as input we can observe that for the four possible input pairs in 
f2i X Qi for a locally defined bivariate operation there can be maximally four 
different subsystems where that function can be evaluated. We can say that 
the values will be distributed over the subsystems. Such an operation is called 
transjunction. Below we give an example of a “conjunctional” transjunction - 
we just display the truth table 





To 


To 


To 


Ta 


F0 


To 


T^ 


Fs 



Every transjunction can be described by such a table or “T-F-pattern” to- 
gether with the indices {a, /3, 7 , <5}. If {a,/?, 7 , <5} ^ {i} then we obtain a tran- 
sjunction - in the previous example we have the type of a conjunction as can 
be seen by omitting the indices. If {a,/?, 7 , <5} = {i} then we have a classical 
conjunction remaining in subsystem Li. In work cited below, we introduced gen- 
eralized transjunctions having more than 4 input pairs. The example which we 
present in section 3 deals with such a generalized trunsj unction. 
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We applied transjunctions for the logical control of cooperating robots sce- 
narios. Cf.[Pfa97] for a brief discussion of such an application. For further infor- 
mation on the subject we refer to [PSS96a], [PSS96b]. In the framework of this 
contribution here we apply methods from logical fiberings to support logically 
the modeling of the aircraft positioning system as described subseqently. 

Generally spoken, we consider the concept of logical fiberings as a natural 
logical modeling approach for multi-agent systems (MAS). This has been dis- 
cussed in [Mei99]. Future work is planned, especially with respect to an extended 
generic airport model using MAS techniques. 

3 Development of an aircraft positioning system 

3.1 Design concept of the positioning tool 

First of all, we want to give a short problem description and explain what we 
mean by aircraft positioning. The main task of aircraft positioning is to find 
schedules for all incoming and outgoing aircraft at an airport. Difficult con- 
straints have to be considered, like runway crossings, (optimal) passenger con- 
nections, aircraft types, special flights, particular gates, airline requests, security 
problems, and others. The main problem in the field of aircraft positioning is 
to find an optimal schedule for all incoming and outgoing aircraft with respect 
to their positions and corresponding gates. Three subtasks have to be distin- 
guished: long-time, short-time and the actual day scheduling. To each subtask 
corresponds an individual knowledge about the aircraft which influences the po- 
sitioning. The knowledge changes rapidly during scheduling. The external state 
of the airport can change rapidly too, caused by construction work, for exam- 
ple. Thus an important design objective is to build a tool which is able to react 
to rapidly changing situations. As previously mentioned, we decided to choose 
logical fiberings as a logical modeling approach. This decision was naturally mo- 
tivated by our problem analysis. Figure 1 shows the global system design. 

Our system includes various agencies and data areas, namely the kernel 
agency, the airport agency and the external data areas, like airport database, 
temporary fiightplan. Another part of our system is the output unit and the out- 
put communication unit, which will be used to visualize an airport utilization. 

The communication between the various parts of our model will be handled 
by a negotiation protocol. This allows us to handle the basic communication in 
our system for the short-time planning task. An augmented final version of the 
tool will be able to treat the two remaining cases too, namely long time planning 
and the actual day. 

3.2 External data areas 

The external data areas are specialized data storages for, e.g. airport description, 
flight characteristics, airport characteristics , airline characteristcs. We use an 
airport database, which includes basic airport information like the number of 
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Fig. 1. Global system design 
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positions and gates. Besides that there are the infoserver, which is our main data 
source for flight information, the airline preferences, the flight specialities and 
the gate and position specialities. All these areas contain positioning relevant 
data which are necessary to calculate an optimal solution for the positioning 
problem. 



3.3 Airport Agency 

The airport agency is used to model a virtual airport. The virtual airport model 
can be applied to an existing airport, in our case airport Frankfurt/Main. The 
design of the virtual airport is based on the concept of logical flberings. The air- 
port agency, respectively the virtual airport, consists of serveral units. Two units 
are interfaces for communication with the enviroment and all others are internal 
units. First, there is the precalculating Altering hierarchical list of positions and 
gates (p.f.H.L.P.G) and second the output communication unit (O.C.U) which 
will not be discussed in this contribution. These two units manage the data 
transfer with the enviroment and prepare input data for the internal use in the 
airport agency and the kernel agency which will be decribed later in this contri- 
bution. Figure 2 gives an overview of the design of the airport agency with their 
units and the internal communication paths. 



LVG Kernel FI19M 

Pr«fereiK«8 Agency Speeiatties 




Fig. 2. Design of the Airport Agency 



The airport agency, shown in Figure 2, includes a number of virtual clusters, 
e.g. ’’AWEST”, ”AOST” or ’’BWEST”. These clusters include the fibers of our 
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airport. We use two different types of fibers one for positions and the other one 
for gates because they have a lot of different characteristics. Some typical char- 
acteristics of a position fiber are the maximum valid wing code or the buffertime, 
on the other side some typical characteristics of a gate fiber are the maximum 
number of allowed passengers in the gate area or the time to bring the passengers 
to the aircraft. 

In terms of logical fiberings to each “agent” corresponds a “local” fiber. These 
fibers are connected via communication paths with the p.f.H.L.P.G. and they can 
have further connections with their neighbors. The connections (communication) 
with neighbors are modeled by transjunctions. A transjunction can be used to 
control the state spaces of neighbored (connected) fibers. A special effect in an 
application of a transjunction can be described as follows: if a local fiber A 
corresponds to, for example, an aircraft or group of passengers (GoPax), and 
there is a transjunction from A to a fiber B, then the state space of fiber B will 
be downgraded by the transjunction. This downgrade of the state space of fiber 
B will be cancelled again as soon as fiber A is no longer used by ( “attached to” ) 
an aircraft or GoPax. 

Example: TVansjMncffon rule ’AlO’ to ’A12’ 

if 11 then 5 

The effeet of this transjunetion is, that the state spaee of ’A 12’ will be down- 
graded to SWC 5, if the eurrent SWC of ’AlO’ is 11. 

Here SWC is the short notation for “wing eode” . One has to take into aeeount 
that different wing eodes have to be distinguished - there exist priorities whieh 
must be taken into eonsideration. 

The eomplete transjunetion eorresponding to this example has 3 truth values 
in the loeal fiber A and 12 values in heal fiber B and therefore represents a 
generalized transjunetion in the sense of seetion 2. The eomplete truth table of 
the transjunetion is displayed in [FW99]. 

Figure 3 shows a special cluster with its communication paths and neigh- 
borhood relationships. Moreover, the complexity of the state space of an agent 
depends on the number of allowed aircraft types for that agent. 

3.4 Kernel Agency 

The kernel agency includes currently 4 different algorithms, the adaptive long- 
time scheduling algorithm, the genetic short-time scheduling algorithm, the ran- 
dom scheduling algorithm, the conflict solving algorithm. Another part of the 
kernel is the kernel communication unit. In this contribution we discuss the 
genetic short-time scheduling algorithm. Treatment of the other scheduling al- 
gorithms is planned as future work. 



Genetic short-time scheduling algorithm This genetic algorithm [Mic96], 
[Hof96] is designed to solve the problems occurring in short time scheduling. It 
uses an already pre-optimized season plan (generated by a human operator or by 
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Fig. 3. Internal structure of cluster ’AWest’ 




Fig. 4. First attempt 



a long time scheduler) together with actual changes and additional information 
(including, among others, actually flying passengers, planes available, passengers 
that need to get a connection). We decided to use evolutionary computing to 
solve this problem. 

The original idea was to treat the positioning problem with a usual genetic 
algorithm, encoding the positions of a given aircraft in a standard way (figure 

4 )- 

Although this approach works it causes a crucial problem: almost no valid 
(valid for a final solution) individuals are created this way since most solutions 
will position one of the planes either on an impossible position (plane is too 
large, for example) or will put planes at the same time on the same position. 
Of course, such solutions would get a very low fitness, so they are not likely to 
produce offspring. Still we get the problem that the resulting algorithm spends 
most of the time searching for a solution where the planes actually fit and not 
for an optimal positioning of the planes - thus wasting valuable time. 
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Fig. 5. Second attempt 



Therefore, another approach was chosen. First we changed the coding such 
that the aircraft are mapped onto positions (figure 5). 

This enables us to use the complete rule set which is provided by the airport 
agency. So we are now able to allow only such genetic operators which produce 
a valid individuum. For example, if the aircraft number one would be a B747 
and the only positions which were allowed for such a plane would be position 
one, two, and three, then a mutation on plane number one could only mutate it 
towards position one, two, or three. 

This second model works very well with the mutation operator, but new 
problems arise when using a crossover operator. The loss of data or the duplica- 
tion of data can happen. These problems are well known since they also occur 
when trying to find solutions to a traveling salesman problem with the help of 
GAs. 

When doing a crossover not only the parts that are actually selected are 
exchanged, but also other parts which are necessary to maintain consistency. To 
this end we have to check for each exchanged plane whether its “counterpart” 
is also moved. If not, we have to set this aircraft also on the exchange list. On 
first view this seems to be a very useful trick to solve the problem, but it also 
produces a problematic side effect. Every time we wish to do a crossover at a 
certain point we also exchange planes on quite randomly chosen other positions. 
This can destroy building blocks, especially in a problem as big as the positioning 
task. For this reason we decided not to implement a crossover operator in the 
classical way. 

The Condense Operator (Mutation 1) This is the simplest mutation operator 
which processes all planes that are not yet at valid positions. It aims at posi- 
tioning such planes correctly. 

The Replaee Operator (Mutation 2) This second mutation operator is almost 
as simple as the first one, but works in the opposite way. It is mainly used to 
maintain the diversity in a population. To do this it moves a plane from one 
position to another one. Technically this is also done in three steps. First one 
aircraft is randomly selected. This aircraft is moved to the Temp place. Temp 
denotes the set of planes not yet positioned. Then the algorithm selects randomly 
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a position where the plane fits. Then the selected plane is force-positioned on 
this selected position. It will be positioned there in any case removing disturbing 
planes if necessary. Later the removed planes will be reordered using the random 
scheduler. 

Chromosome Repositioning (Mutation 3) This is the most ’advanced’ mutation 
operator that will be used in our implementation of the GA. It does not operate 
on single planes like the previous ones, but on complete positions. 

Guided Replaee ( Crossover 1) As already mentioned, many difficulties can arise 
when using crossover operators in a given problem. Therefore we decided not to 
use them in a classical way, but in form of a new class of genetic operators. This 
class works like a mixture of crossover and mutation. It behaves like a normal 
crossover functionally, but it has a probability of occurrence like a mutation. 
So this kind of crossover operators can be considered as guided mutations. The 
Guided Replace operator is the simplest case of an operator of this type. 

Guided Chromosome Repositioning (Crossover 2) This crossover-like operator 
has similarity to the operator “Mutation 3” . The idea is to make a transfer of a 
perfect distribution of planes corresponding to a certain position in an individual 
to another individual. The exact algorithm becomes a rather complicated ruleset. 
A detailed description can be found in [FW99]. 

Cluster Crossover (Crossover 3) This is the most advanced crossover operator. 
Since the airport contains groups (clusters) of positions it is plausible to use 
these in the optimization process. The internal algorithm of the operator has 
close links with the second crossover operator. In fact, we can use this operator 
for the cluster crossover if we select more than one position (all positions in the 
given cluster). 

Fitness Funetion To find an appropriate fitness function was one of the main 
problems of the complete optimization process. We designed the function in close 
cooperation with the experts from Lufthansa. The optimization process is influ- 
enced by many factors which have to be taken into account. Furthermore, the 
approach should be flexible and it should be possible to cope with an optimiza- 
tion task which depends on selected factors only. The complete fitness function F 
consists of several constituents. F{i) is based on four different main optimization 
criteria with respect to an individual i: 

F{i) = P{i) + C{i) + S{i) + Q{i). 

P{i) denotes the part that tries to optimize the number of passengers. C{i) is 
responsible for reducing the connecting time. S{i) describes the service aspects 
and Q{i) the quality of the solution. 

The Greek letters in the subsequent formulas denote parameters which can 
be tuned problem dependent. 
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P{i) = a*IP + !3*OP + 'tP*IB + 5*OB + e*IY + <j)*OY 



The passenger part takes care that a maximum amount of passengers can 
leave or enter a plane directly from the gate {IP and OP). Additionally, also 
the yield of passengers can be taken into account {lY and OY). P{i) models the 
request that a maximum amount of aircraft are well positioned right in front of 
the building {IB and OB). 



C{i) =-f*CT + ri*GD. 

The connexe (connecting passengers) part optimizes the time needed to get 
from one plane to the other. To this end two options are possible. Either the 
buffer time (maximal possible time to reach the plane - walking time) is maxi- 
mized {CT) or special gates are used for planes with many connecting passengers 
{GD). 



S{i) = l*BN + k*BY + X* PD. 

S{i) represents the service part of the fitness function. Three factors are 
summarized here. BN and BY reduce the amount of bus transfers or the amount 
of bus transfers for valuable passengers, respectively. 

The third factor, PD, makes the time between boarding and take off as small 
as possible. 



Q{i) =11* PG + V * GG + TT * GG. 

Finally, Q{i) is responsible for the fact that the solution is admissible at all. 
PG and GG model the constraints that all positions and gates are valid. GG is 
responsible for the request that all connecting passengers are able to catch their 
planes. 



4 Test of the tool in a real world scenario 

In [FW99] a real world example is presented which corresponds to a typical sce- 
nario to be handled every day by Lufthansa and FAG (the Frankfurt airport 
operating center). The initial input to our system is a concrete schedule elabo- 
rated by the long-term scheduling team of FAG. The syntax of the input data 
is of the following type (example) - we display three lines only. 

inbound LH 00201 sta 07:45 in_type I in_fiight_type P inpaxcode 3 
outbound LH 03720 std 08:40 out.type S outpaxcode 3 out jiight.type P 
dest VIE air.type A321 swc 11 pos V123 out_gate B13 
The complete input comprises about 1200 such 3-line units. Evaluating the 
performance of the tool in its application to a real problem situation it turned 
out that the system found better solutions than the human expert. In order 
to produce an optimized solution to the position scheduling task the human 
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operator needs about one week, whereas, working on the same task, our system 
needs about one hour (on a standard modern PC) and it produces even better 
positioning plans. More details and an example of an airport resource utilization 
plan produced by the tool can be found in [FW99]. 

5 Conclusions 

In the previous sections we presented work on the hard problem of optimal 
positioning of aircraft at an airport. Many real world constraints have to be con- 
sidered. The original problem has been described by Lufthansa AG focusing on 
airport Frankfurt/Main. We developed a general (generic) airport model using, 
among others, the concept of logical fiberings. The optimization problem was 
treated by modified and extended genetic algortihms. On the basis of these ap- 
proaches an aircraft positioning tool was developed and implemented, especially 
tailored for computing positioning configurations of the airport Frankfurt/Main. 
A prototypic first version of the system is currently being tested with Lufthansa 
at Frankfurt airport. Our new system achieves much better performance than 
the methods applied before. 

In addition to the previously described methods, in future work we intend 
to use also methods from artificial neural networks (for modeling position con- 
straints and optimization) , make systematic applications of multi-agent systems 
techniques and rule based systems. 
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